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ABSTRACT 



Image understanding for a mobile robotic vehicle is an important and 
complex task for ensuring safe navigation and extended autonomous operations. The goal 
of this work is to implement a working vision-based navigation control mechanism within 
a known environment onboard the autonomous mobile vehicle Yamabico-1 1 . Although 
installing a working hardware system was not accomplished, the image processing, model 
description, pattern matching, and positional correction methods have all been 
implemented and tested on a graphics workstation. A novel approach for straight-edge 
feature extraction based upon least squares fitting of edge-related pixels is presented and 
implemented for the image processing task. A simple method for determining the camera's 
location and orientation {pose ) follows by matching the vertical extracted edges from an 
image with the linear features of a two-dimensional view of the modelled environment 
based upon an estimated pose of the robot. Image processing, construction of the two- 
dimensional view of the model, and pose determination are conducted sequentially in less 
than one minute for a 646 x 486 pixel image on a 35 MHz processor. The pose 
determination results have been tested to be accurate within a few inches for translational 
error and within one degree rotational error. 
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L INTRODUCTION 



A. MOTIVATION FOR VISION DEVELOPMENT 

Image understanding has been a difficult subject in the fields of artificial intelligence and robotics for 
nearly twenty years. The prospect of developing a novel and revolutionary approach to image processing and 
image understanding in the course of writing a master’s thesis seems unlikely. Instead, as with most all vision 
systems developed, this approach will be directed towards developing a visual navigation system based upon 
the specific needs of the robot and tailoring the application to the domain of its operating environment. 
Certain assumptions will be made to simplify 7 the problem. These assumptions are: 

- the robot will operate in an indoor, orthogonal environment, 

- the floor will be a flat surface, 

- the floorplan of the environment will be known, 

- the robot will always be in an upright position, 

- and approximate position information will be accessible. 

The image understanding techniques to be described have been developed utilizing a charge-coupled 
device (CCD) television camera, a graphics workstation, and a video frame grabber. The CCD television 
camera (JVC model TK870U) is capable of providing digitized RGB formatted color information (red, green, 
and blue components) to an image processing system. The graphics workstation is a SiliconGraphics Personal 
Iris and the video frame grabbing board is SiliconGraphics’ VideoFramer. The SiliconGraphics graphics 
library’ supports basic display functions in the C language. The vision based routines described within this 
thesis are programmed in the ANSI C language. 

B. AUTONOMOUS MOBILE ROBOT SYSTEM YAMABICO-11 

Yamabico-11 (Figure 1.1) is an autonomous mobile robot that has been the focus of many students’ 
theses at the Naval Postgraduate School. Work in the fields of motion control, safe path planning, and sonar- 
based navigation are continually ongoing to upgrade the robot’s capabilities. Yamabico’s platform is an 
aluminum cart that stands 36 inches tall, with two main wheels for motion and steering and four smaller 
wheels for support. Its maneuvering capabilities resemble that of a tracked vehicle (it can pivot in place). 
Power is provided from two 12-volt wheelchair batteries. 

Process control is conducted by one Motorola 68020 32-bit microprocessor with an accompanying 
Motorola 68881 floating point coprocessor. The processor has 1Mbyte of onboard RAM and runs at 16MHz. 
Control information is passed over a VME bus which also carries a dual axis controller board and two 4- 



1 



channel serial communication boards. The dual axis controller interfaces with driving and braking motors. 
The serial channels interface with an onboard terminal and a modem to a host computer. 

Its only sensors, twelve ultrasonic sonars that operate at 40kHz, are mounted about the lower base and 
are controlled by three Motorola 6809-based microprocessors. The Sonar Control Language [SHE91] and the 
Mobile Motion Language (MML) [KAN89a] provide easy sonar and motion control features to the 
programmer and are downloaded to Yamabico via an RS232 serial port from the host computer, a Sun 
Microsystems 3/160. The system is composed of a kernel and a user-program . The kernel (about 100 kbytes) 
only needs to be downloaded once. The user-program, user(), specifies Yamabico’s tasks. Once the object 
code has been downloaded and the serial link disconnected, all computation is performed autonomously and 
is executed via a keyboard mounted on top of Yamabico. 

Presently, Yamabico can navigate its way through a “known” corridor by utilizing ultra-sonic range 
information to update its position which is maintained by wheel motion dead-reckoning. Yamabico’s ultra- 
sonic sensors are effective in ranges from 9.6cm to 400cm and are accurate to 18.95mm at a range of 500mm. 
The range information can directly update the current position in a known environment (such as our testing 
grounds, the fifth floor of Spanagel Hall at the Naval Postgraduate School). Prior to the work outlined in this 
thesis. Yamabico had no image based sensor capabilities at all. The purpose of this thesis is to develop and 
implement a vision based sensor system for Yamabico- 11 suitable for navigation and exploration of its 
environment. 




Figure 1.1 The autonomous mobile robot Yamabico-] 1. 
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C. LITERATURE SURVEY 



Image understanding has been a complex topic for computer scientists for almost two decades. The 
ideas and algorithms utilized in this work stem from the image processing, linear filling, pattern matching, 
and pose determination methods that have been found to be successful in past applications. Two good sources 
for the basic ideas necessary in image understanding are by Ballard [BAL82] and Nevada [NEV82]. These 
are both textbooks that outline the more commonly used methods and the background fundamentals. They 
cover a wide range of topics including low level image processing, graph-theory for pattern matching, and 
model-base object recognition. 

A paper by Mari [MAR79] outlines requirements for model-based image understanding and provides 
insight for construction of suitable models, similar to his “2D+,” model for vision applications. Stein’s thesis 
[STE92] describes the cmironment model developed for the vision navigation method developed in this 
thesis. Two papers by Kanayama, (KAN90a] and [KAN90b], were consulted for familiarization with the 
robot Vamabico's motion control capabilities (with the MML language) and the application of linear fitting 
of sonar point data b> least squares for environment feature mapping. Sherfey’s thesis [SHE91] provides the 
details on Yamabico’s sonar capabilities. 

A lot of recently published research conducted at the University of Massachusetts at Amherst provided 
motivation to develop 1 aster and simpler image edge extraction and visual navigation routines. Papers from 
this group by Beveridge [BEY90j, Kumar (KUM89], and Fennema (FEN90] describe methods for pattern 
matching, pose determination, and their experimental results. Edge extraction w'as conducted via the Hough 
transform and pattern matching performed by random combinations of image to model line matches. 

Pose determination methods described b\ Haralick [HAR89] and Liu [LIU90] are algorithms based 
upon the general trigonometric principles of photogram metry. A phologrammetry textbook, such as Moffil’s 
[MOF80], proved to be a good source to explain these fundamentals. During development of this algorithm, 
a means lor pose verification was found to be necessary. Methods tested by Heller [HEL89] show the results 
of verifying all image lines w ith model lines and the results of verifying model lines wdth edge-related pixel 
data. A basic computer graphics textbook by Hall [HAL89J was also consulted for checking video source 
formats and computer image data formats. 



D. THESIS ORGANIZATION 



The following Chapters outline the development of the algorithm applied towards a simple visual 
navigation scheme for an autonomous mobile robot in a known orthogonal environment. The objectives are 
stated in Chapter n. Chapter HI presents the general method of feature extraction based upon determining the 
edges depicted in an image. Chapter IV discusses the method in which the image’s pixels are grouped into 
two-dimensional regions describing the edges. Extraction of straight line segments from two-dimensional 
areas based upon least-squares fitting of datapoints is presented in Chapter V. The straight edge extraction 
algorithm is then outlined in Chapter VI. The algorithm’s results and the effects of varying parameters within 
the algorithm are shown in Chapter VH. Chapter VIII discusses implementation factors such as image data 
structures and the environment model. 

Chapter IX discusses the method for pattern matching of extracted edges from the image with modelled 
environment features. Chapter X describes pose (the vehicle’s position and orientation within the 
environment) determination based upon disparities between the image and the expected view of the modelled 
environment from an assumed position. Pose determination results are shown in Chapter X3. Conclusions and 
subsequent topics for this visual navigation system are presented in Chapter XH. 

Appendix A specifies the atan2 function used for determination of a vector orientation and it contains 
the data types header files used. Appendix B contains the edge extraction implementations findedge and 
fasted ge and all the required header files for compilation. Appendix C contains the implementation vertmatch 
which extracts vertical image lines, matches them with the model lines from an expected two-dimensional 
view of the environment, and determines the robot’s correct position and orientation. Appendix D contains a 
user’s guide for the compilation and operation of the edge extraction and pose determination programs. 
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EL OBJECTIVES 



There are two primary objectives for the development of a vision understanding system for the 
Yamabico-1 1 robot. They are similar to the objectives for any autonomous mobile vehicle. 

1. Update correct vehicle position and orientation for precise navigation. 

2. Recognize specific objects using a prior knowledge based upon the task requirements and tire 
operational environment. 

A. POSITION / ORIENTATION IDENTIFICATION IN A KNOWN WORLD 

Determining accurate position and orientation autonomously is a crucial requirement for the successful 
and safe navigation of any mobile robot. This task is also known as pose determination and is used to update 
the robot’s positional information that is maintained by wheel motion dead-reckoning. Small errors from 
dead-reckoning arise quickly and are compounded by turns and acceleration. Continuous periodic pose 
determination is fundamental for precise dead- reckoning based navigation. Chapter X outlines a simple 
position-correction algorithm via pattern-matching of the extracted features from a two-dimensional video 
image and two-dimensional view of mapped features of the three-dimensional environment model 
maintained by the robot. This algorithm is based upon previous works [HAR89], [MAR79], [MOF80], 
[KUM89], [FEN90]. and [LIU90], whose roots stem from shipboard navigation and aerial photogrammetry. 

B. OBJECT RECOGNITION IN AN UNKNOWN OR PARTIALLY KNOWN WORLD 

The second objective is the recognition of specific objects via three-dimensional models. This is a 
much more complex task than the first due to the uncertainty of individual objects presence and location. The 
first task of pose determination is a subset of this task. The operating environment is modelled as a three- 
dimensional object and determining the camera's pose is only determination of viewing aspect of the object. 
Object recognition covers many other topics including mapping an unknown environment, searching for a 
specific goal, classifying encountered objects, and avoiding unknown obstacles. Some aspects of object 
recognition are detailed in [BAL82], [MAR79], [NEV82], and many others. A central theme of all vision- 
based object recognition applications is modelling and searching for only the objects that are expected to be 
encountered and classifying unknown items by their geometrical properties. This objective is not approached 
within this paper, but will hopefully be accomplished for Yamabico by a successor 
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in. METHOD 



A. ARCHITECTURE FOR IMAGE BASED NAVIGATION 

Extracting desired features from an image is a crucial process in any image understanding 
implementation. Some methods focus on determining areas of common light intensity, color, or texture. 
These methods often include segmentation of the image by region growing and are generally suitable for 
images with small areas of pixels with common attributes (e.g. a picture of a forest). Other methods search 
for the edges between contrasting bordering areas of pixels with similar attributes. Likewise, these methods 
are suitable for images with large areas of common light intensity, color, or texture (e.g. a picture that 
included bare walls). This method will be of the latter category; extracting the edge features of an image. This 
method was chosen for recognizing the straight edges of orthogonal objects whose surfaces are often uniform 
in color and texture. The images that the edge extraction methods will be applied to, will be pictures of the 
interiors of offices and hallways. Pictures where straight line segments would be common and suitable for 
describing the objects in the image. 

The goal of the image processing part of the vision system will be to find straight line segments in the 
edges of boundaries between areas of similar light intensity in an image. The straight line segments should 
be a sufficiently simple data structure for use in follow-on image understanding implementations. The 
specific problem of image understanding for navigation of Yamabico in a known, orthogonal environment 
will be split into the following subproblems: 

1. Image Processing - extracting desired features (straight line edges) from the input image. 

2. Pattern Matching - correlating extracted features with known features described in a three- 
dimensional model of the environment. 

3. Pose Determination - calculating the position and orientation of the robot within its modelled 
operating environment. 

Since Yamabico has no vision based sensor capabilities, the following features were identified for the 
implementation of a vision understanding system (Figure 3.1) 

- a three-dimensional model representation of the operating environment, 

- camera image processing functions, 

- image feature extracting functions, 

- pattern matching routines, 

- and visual-based position correction methods. 
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Figure 3.1. The general outline for image understanding on Yamabico-11. 



In order to accomplish the image processing task of finding straight line segments in an image, the 
borders (edges) between areas will be identified, segmented, and then simplified in terms of straight line 
segments (Figure 3.2 ). The identification of the edges in an image (Figure 3.3) is best described by a gradient 
image (Figure 3.4). Two important steps to follow are: 

1. grouping of pixels into contiguous regions of separate edges, and 

2. linear fitting of the edges. 
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GRAYSCALE IMAGE 



I 

Extracting Edge Regions 



EDGES 

I 

Application of Least Squares Fit 
method on Edges 



▼ 

LINE SEGMENTS 





Figure 3.2. Outline of method for edge extraction of an image. Edges will be represented 
as line segments. 



Thus, once the gradient image is made, pixels that define the edges (the black pixels of Figure 3.4) must 
be grouped and then fitted to a straight line segment by least squares fit to the major axis of the edge region. 
Determining when to start and stop constructing separate edges is a crucial subject. Checking the validity of 
tlie line segments will be the final requirement of the image processing task. 
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Figure 3.3. An input image of a chair. 




Figure 3.4 The gradient image for the chair in Figure 3.3. Edges between regions of 
common light intensities are depicted by black pixels. 
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Figure 3.5. The extracted line segments based upon the grouping of similar edge-related pixels. 

The pattern matching process will require two inputs. These will be the linear features from the three- 
dimensional model mapped onto a two-dimensional viewing plane and edges extracted from the image 
processing system. The three-dimensional model is a tw o-dimensional floorplan plus vertical components for 
height measurement. This model will be used to construct a two-dimensional “wire-frame” view' of the 
environment from any given pose for the robot’s camera. This view- will be used for w'hat the robot will expeet 
to see from its present location and orientation. The three-dimensional modelling system for Yamabieo is 
being pursued concurrently with this work by Lt. Jim Stein for his master’s thesis [STE92]. 

The image processing system could be executed in parallel with the three-dimensional modelling 
system as it provides an updated two-dimensional wire-frame view' of the robot’s orthogonal environment. 
The pattern matching process w ill correlate the extracted edges from the image w'ith the knowm features from 
the model. 

The task of pose determination wall follow the matching of the three most significant edges with all 
possible vertical model lines. The observed difference of horizontal viewing angles between the vertical 
image edges will be fitted w ith the map loealions of vertical model lines to determine the only possible 
position and orientation of the robot’s camera. Many different possible values for the robot’s pose will be 
calculated as different line matchings are tested. 
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A method for pose verification will be necessary to select the best possible pose (hence, also the best 
matching ) to be returned as the correct pose of the robot. The correct pose can then be used to update the pose 
maintained by dead-reckoning. Sensor fusion with the present sonar system could possibly follow for 
confirming the accuracy of the visual fix for the pose. 
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IV. GENERATING REGIONS OF EDGES 



A. GRADIENT IMAGE 

The first requirement for the image edge extraction algorithm is the determination of high contrast 
regions between the areas of similar light intensity within the image. These regions of significant light 
intensity contrast will define the edges. The CCD camera simplifies the quantification of light intensity by 
discretization of the image into separate pixels. Chapter VII identifies some of the formats provided by the 
camera and used in the graphics workstation. The pixel format utilized herein is the RGB format which 
specifies the red, green, and blue color attributes of the pixel and is described fully in Chapter VII. Differences 
between the areas of the image are therefore determined by differences in pixel intensities. The simplest way 
to consider pixel inteasities is in terms of grayscale values vice considering the separate associated color 
attributes of each pixel. The color to grayscale conversion of RGB pixels in also described in chapter VII. 
Determining the differences in grayscale intensities between adjacent pixels in a two-dimensional image will 
result in a gradient value for each pixel. The pixel gradient is a vector whose magnitude represents the amount 
of light intensity change between adjacent pixels, and whose angular orientation is directed towards the 
lighter pixel. Pixel gradient values will be determined in a two-dimensional cartesian coordinate plane by the 
amount of change of grayscale intensities in both the horizontal (dx) and vertical (A’) directions of the image. 

1. Gradient Window Operators 

A common method to determine pixel gradients is by the use of gradient window operators. Two 
gradient window operators, one for determining the change of pixel intensities in the horizontal (dx) direction 
and another for the vertical (dy) direction, must be specified. Gradient window operators are square matrices 
of weights, mapped onto a group of pixels about a central pixel (or point). The weights are multiplied with 
the intensities of the surrounding pixels and then summed to provide values for the intensity changes in the 
horizontal and vertical axes. Commonly used operators are the Four-Square, Prewitt, Sobel, and Schirai 
(Figures 4. 1 through 4.4). 
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Figure 4.1. The Four-square gradient windows. 
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Figure 4.2. The Prewitt gradient windows. 
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Figure 4.3. The Sobel gradient windows. 
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Figure 4.4. The Schirai gradient windows. 



To be geometrically correct for a two-dimensional plane of pixels evenly spaced in both horizontal 
and vertical directions, a modified Sobel operator should be used with values of Jl vice 2 for weights of the 
non-diagonal pixels (Figure 4.5). This would assume that all of the pixels sensors on the CCD array in the 
camera are in a perfectly square grid. How ever, since one of the assumptions of this vision system is to operate 
in an orthogonal environment, the standard Sobel operator could prove to be more valuable since it enhances 
the vertical and horizontal edges found in the orthogonal environment. After implementing and testing all of 
these window operators, the Sobel seemed to provide the strongest edges and the least amount of smaller, 
insignificant and undesirable edges . 



1 


J 2 


1 


0 


0 


0 


-1 




-1 



-1 


0 


1 


-72 


0 


72 


-1 


0 


1 



dx dy 

Figure 4.5. The modified Sobel gradient window's. 
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Once the values for dx and dy are calculated by the gradient window operators, the gradient is 
easily calculated. Pixel gradient magnitude (|V {pixel { ) | ) and gradient angle (V a (pixel { ) ) would then be 

determined by equations 4. 1 and 4.2. The atan2( dy.dx) function is common in many programming languages ’ 
standard math libraries and is described in Appendix A. 



<] 


= J(dx i ) 2 + (dy t ) 2 


(4.1) 


V aipixelj) 


= atan2 {dy-, dx { ) 


(4.2) 



To determine if a pixel is part of an edge between two areas of an image, the gradient magnitude 
must be greater than a specified threshold value, C;. Such a threshold can be determined dynamically by 
scanning through the image once to determine the average weight of pixel intensities or by maintaining a 
histogram of previous images’ average pixel intensities. An alternative approach is to keep the gradient 
magnitude threshold value as static value, determined by testing. This method would be suitable for a robot 
operating in an environment with relatively constant illumination and alleviates the requirement of an extra 
scan through the image to dynamically determine a value for Cj. This method will be pursued based upon the 
assumption that the robot will initially be tested in an indoor environment with relatively uniform lighting, 
whereas the other method for threshold calculation would be more suitable for the vision system adjusting to 
major environment lighting changes (e.g. the robot travelling into an unlit room). 



2. Construction of the Gradient Image with the Sobel Operator 

The Sobel gradient window' operators will be utilized to calculate pixel gradients from the 
grayscale values for each pixel. The Sobel operator is defined as a two-dimensional matrix; that is to be 
implemented upon pixels read and stored in a one-dimensional array (see Chapter VIII for description of the 
data structure for the image). Using the image's horizontal dimension (xsize) as an offset for finding the 
adjacent pixels in the neighboring rows of a two-dimensional image, equations for calculating the dx and dy 
values for the gradient of a central pixel. /, are defined by equations 4.3 and 4.4. A pointer ( ptr ). is utilized to 
access each grayscale pixel value from the one-dimensional array in the data structure for the image. 

- ptr [i + xsize - 1] +ptr [/ + xsize + 1] 
dx = - 2ptr [/ - 1] + 2ptr [/ + 1] (4.3) 

-ptr [i — xsize - 1] +ptr [/ -xsize + 1] 
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ptr [/ + xsize - 1] + 2ptr[i+ xsize] + ptr [i + xsize + l] 

dx = 

- ptr [i - xsize - 1] - 2 ptr [/ - xsize ] -pfr [i — jc^/ze 4* 1] 



(4.4) 



Notice that in order to calculate the Sobel operator for pixel z, pixel i+xsize+1 and all previous 
pixels must have a grayscale values already determined. A possible method could be to first construct a 
corresponding grayscale image of the input (color) image followed by construction of the gradient image 
using the Sobel operator equations. Pixel gradient magnitudes larger than the predefined threshold (Cj) would 
then be set to be black in the gradient image and all pixels would be white. This would provide a gradient 
image such as displayed in Figure 3.4. A simple algorithm to do this would be: 

- Read in input image. 

- Allocate memory for new' grayscale and gradient images. 

- Build the grayscale image by grayscale conversion for all pixels. 

- Build the gradient image via Sobel operator for all pixels. 

- If pixel gradient magnitude is greater than the magnitude threshold, C;,then set the 
corresponding pixel in the gradient image black, else set the corresponding pixel white. 

3. When Not to Use the Sobel Operator 

However, not every pixel in the input image (more precisely, the grayscale image) can have their 
associated gradient values properly calculated. The outermost pixels, those in the top and bottom rows and 
the leftmost and rightmost columns will not have gradient values since the Sobel operator requires eight 
adjacent pixels about the central pixel gradient value. Calculating gradient for these outermost pixels will only 
produce errors w'hen using the gradients. Therefore, the dimensions of the gradient image are tw'o pixels less 
in both horizontal and vertical directions and careful consideration must be paid tow'ards scanning through 
the input image in order not to calculate pixel gradients for the outermost pixels. 



B. FINDING CONTIGUOUS REGIONS OF PIXELS TO DESCRIBE EDGES 

Since the black pixels in the gradient image represent pixels with gradient magnitudes sufficiently 
large to be considered part of an edge, each black pixel must be considered for grouping into contiguous 
regions that define the edges. Therefore, grouping pixels into regions will be performed upon every pixel with 
a gradient magnitude larger than the threshold. Cj. Determining a suitable value for C; is critical to the 
success of this algorithm. A threshold that is too high will not allow for the detection of all significant edges 
with those that are found being broken and incomplete. On the other hand, a threshold that is too low lead to 
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edge finding and linear fitting of non-edge areas of the image. Chapter VII shows results of varying the 
gradient magnitude threshold Cj . 

After a suitable threshold is determined and used, the problem of grouping edge-related pixels is to 
define how two pixels share features common to an edge. Figure 4.6 can be considered as a portion of a 
gradient image with two distinct areas of common light intensity (areas A and B) and the subsequent edge in 
between, comprised of pixels with gradient magnitudes greater than the given threshold. In order to describe 
the edge, each pixel must be grouped into one contiguous region. There are three important aspects of a pixel 
that can be used for this comparison: pixel gradient magnitude, pixel gradient angle, and pixel adjacency. 
Pixel gradient magnitude has already been considered by testing with threshold Cj. 



line segment describing 




Figure 4.6. A region of pixels defining an edge in the gradient image. A and B are areas of 
common light intensity (pixels that are white in the gradient image). Area B is darker than 
A, subsequently all edge pixel gradients will be directed towards A. Similarly all edge pixel 
gradient angles will be close to the normal of the line segment describing the edge. 



For two pixels to be of the same edge, they must both have gradient angles that are close . “Closeness” 
can be defined by ensuring the difference between two angles is less than some constant angle, C 2 . The pixel 
gradient orientation is the normal to the tangent of the border between the light and dark areas of the image 
at that point (areas A and B in Figure 4.6). Therefore, all pixel gradient orientations will be close to the normal 
of the line segment describing the edge region. If a pixel’s gradient angle is not close, then the pixel can not 
be considered to be part of that edge. The effects of varying C 2 am shown in Chapter YD. 
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The adjacency requirement can readily be seen in Figure 4.6. For a contiguous region of pixels to 
define an edge, all pixels in the edge must be adjacent to at least one other pixel in the edge. Thus, after testing 
a pixel’s gradient magnitude for description of the gradient image, only the pixel’s gradient angle and the 
pixel’s adjacency with a previously edge -defined pixel needs to be considered (Figure 4.7). If these 
requirements fail, the pixel must still be considered being a member of an edge and therefore it will be the 
first pixel to define a new edge. 




row presently being scanned 



Figure 4.7. Pixel i is only compared with four adjacent pixels to check gradient orientation 
closeness. One of these four pixels must already belong to a defined edge. 



C. DECIDING WHEN TO STOP ADDING PIXELS TO AN EDGE 

Continually declaring new edges for every pixel that does not meet the membership requirements for 
a declared edge will rapidly decrease performance as the number of edges gets large. It is desirable to process 
edges for linear fitting once it has been determined that no more pixels could possibly be included. This is 
done by enforcing the eight-pixel adjacency rule for pixel inclusion in an edge. Every time that the scan of 
pixels starts at a new row in the image, all edges must be inspected to ensure that the last pixel added to each 
was from the row' just scanned (Figure 4.8). If this check fails, then no pixels found on the present row (the 
row to be scanned) could be included with that edge, therefore that edge can be considered completed for 
grouping of contiguous pixels and may be processed for linear fitting. Thus all edges that pass this test are 
active edges that can still accept more pixels for inclusion. 
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row just scanned 



Figure 4.8. Prior to starting the scan of a new row, the list of edge regions must be reviewed to ensure 
that the last pixel added to each edge occurred in the row just scanned. If not, pixels found on the new 
row could not possibly be associated with the edge. Thus, edge A would be removed from the edge 
list whereas eage B is retained as an “active” edge that coulcf still accept more pixels for inclusion. 
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V. LINE FITTING BY LEAST SQUARES 



A. LEAST SQUARES FIT METHOD 

One method for determine a line segment that adequately describes a two-dimensional region of 
datapoints is by least squares. Linear fitting by least squares is a simple and efficient method for calculating 
thousands of datapoints. In an image, the least squares fit moments , described by Kanayama [KAN90b], of 
pixel locations (x,y pixel coordinates of the image) for all pixels in the edge region will yield an associated 
line segment for the region’s major axis. Verification of the region is required to ensure that the least squares 
fit moments are representative of a line segment. Regions that meet the requirements for a line segment are 
saved for the desired output and all others are discarded. 

Once the decision has been made to include a pixel as a member of an edge region, the least squares 
fit moments must be updated for the region to include the new pixel. Consider the image as a cartesian 
coordinate system, discretely numbered by pixels in both horizontal and vertical directions. Each edge region, 
/?, can be considered as a collection of pixels occupying a two-dimensional space (Figure 4.6). Pixels’ 
positions (numbered by pixels in x and y axes) are used to define the least square fit moments m^ 

1. Primary Least Squares Fit Moments 



Each moment nij k (0 <j\ k < 2 and j + k < 2) of an edge region, R, is defined as: 



n 




(5.1) 



Considering that m 00 is equal to n, m 00 defines the number of pixels associated with region, R. 
The centroid C of an edge region, R, is defined as 




(5.2) 



2. Secondary Least Squares Fit Moments 



The secondary^ moments about the centroid are: 
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3. Parametric Line Fitting of the Principle Axis 

Using a parametric representation (r,a) of a line L, a point at coordinates (x,y) satisfies the 

equation 



r = x cos a + vsrna 



-71 K 

( t <o<2>. 



(5.6) 



The residual of pixel / (located at (.v r v f )) and the line L, defined by 8,, is the distance perpendicular 
to L and pixel i. 

8.= (n x -.\j) cosa+ (|i x - y-) sina. (5.7) 

The projection of pixel /, onto the line L is 



Pi = (a\ + 8-cosa, v £ - + S.sina) . (5.8) 

Therefore, p' j and p' n% the projections of the first and last pixels associated with the edge region R onto the 

line segment L, can be used as estimates of the endpoints of L. 

The sum of all the residuals is then 

n 

S = (/ -A^cosa-y.cosa) 2 . (5.9) 

i = 1 



The line that best fits the set of points will minimize the sum S. Thus, the optimum line L (r, a) must satisfy 



dS _ dS 

dr da 



(5.10) 
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Therefore, 



dS 



— = 2 V (r-A,cosa - v.sina) 
dr ' " 



i = i 



= 2 






= 2 {rwQo - m 10 cosa - m 01 sina) 
= 0 



and r can be expressed as 
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(5.11) 



( 5 . 12 ) 
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where ; may be negative. Substituting r in equation 5.9 by equation 5.12, 

n 

S = X < ( x i ~ M. v ) cosa + O,- “ My) sin a) 2 . 



(5.13) 



i = 1 

Finally, dS/da. can be solved for. 



dS 



da 



- = 2 ( (•*, - M A ) cosa + (v,--|A v ) sina) • (- (a,-- fx^) sina+ {y.- n ) cosa) 



< = l 



2 ( O’l - My) 2 - O', - M,) 2 ) Sinacosa 
/ = 1 
n 

+ 2 Ov - M. t ) (>’,• - My) ( ( cosa) 2 - ( sina) 2 ) 



/ = l 



= o (^ 02 _i ^ 2 o) sin 2 a + M 11 cos 2 a = 0 

Which provides a solution to the normal of the line L, a. 

1 ( 2 M 



a - - atan 



n 



2 v ^02 ^20 

Equations 5.12 and 5.15 are the solutions for the line parameters generated by a least squares fit. 



(5.14) 



(5.15) 
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B. 



TESTS FOR VALIDITY OF AN EDGE REPRESENTING A LINE SEGMENT 



The equivalent ellipse of inertia (Figure 5. 1 ) for the edge region, R, will have the same moments about 
tire centroid (M A/ 02 ) ^ ^ major ^ M mini)n the moments about the major and minor axes of 

the ellipse, are defined as: 



AT 



major 



(M 2 o + A/o 2 ) (M 02 Af 20 ) 2 
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( 5 . 16 ) 
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(5.17) 



The lengths of the major and minor axes, d major and d mirwr , are defined to be: 
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Figure 5.1. The major and minor axes of the equivalent ellipse of inertia for the 
region of pixels defining an edge. 
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For this implementation of least squares fit, the major axis will be considered to be the associated line 
segment for the edge region. The endpoints of the major axis will be the endpoints for the desired line 

segment. The projections of the first and last pixels, p' ^ and p' n , associated with the region/?, will therefore 



be used as estimates for the two endpoints of L. A ratio, p, of the axes length can then be used to describe the 
thinness of the ellipse. 



p = 



d„- 

minor 



major 



(5.20) 



Using the values for number of pixels {m 00 ), major axis length (d^j^), and ellipse thinness (p) as 
parameters for comparison to edge significance, edge length and how much the edge resembles a line can be 
simply tested as long as the least squares fits moments are maintained for every pixel included in an edge. Let 

C 3 ( 0 < C\ < 1 ) bea constant for the maximum allowable ratio p that can be used to described a line. Let 
C 4 be a specified constant for the minimum number of pixels and Cy for the minimum line length. Three 
requirements can therefore be specified for the line testing of an edge region. 

1 . the ratio of axes lengths less than the maximum ratio (p < Cj), 

2. the number of pixels greater than the specified minimum (moo > C#)» and 

3. the line length of the region greater than the minimum length (d ma j or > Cy). 

The ratio p proves to be the most significant measurement. For p to equal 1.0 means that the length of 
the minor axis is equal to the length of the major axis, representing an edge region that resembles a circular 
blob. Therefore, p can be compared to a maximum ratio, Cj, specified by the user. Cy equal to 1.0 allows all 
edge regions to be considered thin enough to represent a line segment. A value of 0.1 seems to work well. 

The two other tests can be used to trim down the number of smaller, less significant edges found. 
Specifying minimum number of pixels in an edge and minimum edge length, C 4 and Cy respectively, is an 
effective means to filter out less significant line segments. The regions that meet these requirements will be 
saved for the desired output as the found line segments and all other regions will be discarded. The effects of 
varying the parameters Cy, Q, and Cy are described in Chapter VII. 
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VI. ALGORITHM FOR EDGE EXTRACTION 



From the principles stated in the previous chapters, the following elements were identified to be 
incorporated the edge extraction algorithm: 

1. Determine which pixels compose the edges. 

2. Group pixels together into regions describing individual edges. 

3. Describe a completed edge region in terms of a line segment via least squares fit of pixels’ 
positions. 

4. Ensure that line segment representation of the edge meets the specified thinness and length 
requirements. 

The implementations are coded in the ANSI version of the C language and are contained in Appendix B. A 
user’s guide for compiling and executing the implementations is in Appendix D. 

A. THE FINDEDGE METHOD 

Findedge is a C-program that follows the above oudine. Remembering that the grayscale value for 
pixel Z+xj/re+i must be determined prior to the gradient calculation for pixel /, the algorithm for findedge 
can be stated to build both the grayscale and gradient images in one pass of scanning the input (color) image. 
This algorithm will ensure that gradient values are not calculated for the outermost pixels. Pixels with 
gradient magnitudes greater than the specified threshold, C ; , will be considered as datapoints for the edges. 

Two tests were identified for a an edge-related pixel to be included as a member of a particular edge. 
Those tests were: 

1. The pixel must be adjacent to a pixel already included in an edge. 

2. The pixel's gradient angle must be “close” to that of the averaged angle of adjacent pixel’s. 

Two angles are “close” if the difference between them is less than a maximum constant, C 2 . These tests 
can be conducted immediately once a pixel has been identified to be an edge-related pixel. Therefore, ah 
black pixels in the gradient image will be tested for edge membership. 

Pixels meeting the edge membership requirements must then be incorporated with the edge by 
updating the least squares fit moments, rrij h for the edge region, R . This is accomplished by determining the 
;th pixel’s two-dimensional image position (x,y) and summing the moments, mj h with the corresponding 
weights. If a pixel fails the membership test for the declared edges, then that pixel will be used as the first to 
declare a new edge. Thus, a linked list is the preferable data structure to maintain the declared edge regions. 

Observ ing that new edges are declared as a linear scan of the image is conducted, and that the first 
requirement for pixel membership with an edge is adjacency, a pixel needs only to conduct the membership 
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test with the pixel to the immediate left (assuming a left-to-right, bottom-to-top scan). This builds edge 
regions for the present row only. Prior to continuing the scan at the start of the next row, the list of edge 
regions for the row just scanned will be compared with the list of edge regions from the previous row to see 
if they can be combined based upon vertical adjacency of edge regions between the two rows and “closeness” 
of average orientation of pixels’ gradient angles. 

Maintaining values for the first and last pixels included for each edge is necessary for the determination 
of line segment endpoints. Keeping track of first and last pixels added is straightforward with only one 
exception for horizontally oriented lines. The case for horizontal lines (Figure 6.1) requires determination of 
endpoints based upon the leftmost and rightmost pixels of the edge vice the first and last pixels added. 



General Case of Determining Line Segment Endpoints 
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Horizontal Case for Line Segment Endpoints 
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Figure 6. 1 General and horizontal cases for determining pixels to be used for calculation of 
line segment endpoints. The first and last pixels in the horizontal case may not necessarily 
fully describe the extent of the associated line segment, in which case the leftmost and right- 
most pixels should be used. 
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Edges from the previous row that are not appended with pixels from the row just scanned are 
considered finished with their construction and can therefore be tested for meeting the three requirements for 
a line segment. These requirements were identified to be: 

1 . The ratio of lengths of major and minor axes (p) < C 3 . 

2. Number of pixels in an edge (m 00 ) > C 4 . 

3. The length of the major axis ( d major ) > C 5 . 

Determining d maj - or and p is easily done since the primary least square fit moments are updated for 
every pixel added. Edges that meet these requirements will serve as the desired output of straight edges, 
otherwise they are discarded and the associated memory is freed. 



B. AN ATTEMPT TO CUT COMPUTATION COSTS 

The findedge algorithm requires an averaged value of the pixel gradient angle orientations to be 
maintained for each edge. This average value is used for the comparison with pixel gradient orientation to 
determine angle “closeness." Thus, for every pixel added to an edge, the average of the pixels’ gradient 
orientation must be calculated. To alleviate these computations, the specified maximum value for angle 
closeness, C 2 , could be used to determine two static sets of angles that any pixel gradient orientation could be 
defined by. These two sets could be thought of as dividing up all possible angles between n and -n by C 2 as 
illustrated in Figure 6.2. 

“Closeness” could now be defined by comparing the two groups that a pixel gradient angle belongs to 
with the groups of the adjacent edge-related pixel’s gradient angle. If either groups are similar, then the 
gradient angles are close. This method is termed static since the two sets of angles are computed only once 
and remain unchanged whereas the findedge method continually recomputed the average gradient angle of 
the pixels in the edge region. The results of this method of pixel grouping into straight line edges are shown 
in Chapter VII. Compared with the line segments from the findedge algorithm, more line segments are 
extracted but shorter and more fragmented. Since the desired output is a simple structure (line segments) that 
best represents the linear features in an image, the performance of the static method is worse. 
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Division of angles by C 2 = 20.0 degrees 



Angle Group 1 



Angle Group 2 




Figure 6.2. The two static groups of angles determined for C 2 = 20.0 degrees. Each pixel 
gradient orientation will belong to one region from each group of angles. The groups can 
be thought of as overlapping in such a way so that “close” gradient angles not belonging 
to the same region in one group will belong to the same region in the other group. 



C. IGNORING GRAYSCALE CONVERSION WITH THE FASTEDGE METHOD 

Computation costs can still be reduced in th efindedge algorithm by omitting the construction of the 
grayscale and gradient images. Observing that the green component of the RGB color format accounts for the 
majority of grayscale weights, pixel gradients can be estimated by using only the green component value. This 
is accomplished with a bitmask to get only the green color values for use with the Sobel operator (equations 
4.3 and 4.4). Thus, construction of the grayscale image is unnecessary. Similarly, by observing that the 
gradient image only identifies those pixels that describe edges, and otherwise not required for the construction 
of the edges, its construction and storage as a real image will be omitted. 

Therefore, the program fastedge can perform the same straight line edge extraction as findedge ; 
however, since the values for pixel gradient calculation are only the green color components and not the 
weighted grayscale sum, a new gradient magnitude threshold value (C ; ) must be determined. The fastedge 
implementation displays the input (color) image and draws the extracted lines over the image. Results are in 
Chapter VII. A guide for compiling and using fastedge is contained in Appendix D 
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VII. IMPLEMENTATION 



A. SYSTEM DESCRIPTION 

The graphics workstation used for feature extraction is a SiliconGraphics Personal Iris. Its graphics 
support libraries are implemented in the C language: therefore, the code for the findedge and fastedge methods 
is in C. The VideoFramer frame grabbing hardware is a daughterboard on the Iris that accepts the pixel 
information after it has been converted from the video source signal to an acceptable pixel format (e.g. RGB). 
A typical image from the charged-couple device (CCD) television camera is 646 x 486 pixels. That is 3 13,956 
total pixels to be stored in a file approximately 1.6 Mbytes. Efficient algorithm design is paramount for such 
large image files. The Personal Iris is a 35 MHz machine that can read an image file and conduct edge 
extraction via the fastedge implementation in approximately 15 seconds (about 3 seconds for an image 
“shrunken" to half of its dimensions). 

B. THE INPUT IMAGE 

Prior to encoding an algorithm to find the straight line edges in an image, a careful analysis of the 
image must be considered. The CCD camera used (JVC model TK870U) can provide a video signal in various 
formats. Some of these video formats include: 

- Betamax Video 

- D1 Video 

- NTSC {National Television Systems Committee format) 

-PAL 

- RGB (red, green, blue attributes) 

- Super VHS Video 

There are also various formats for saving and displaying images as files on a computer. Some of these 
formats include: 

- Apple Macintosh MacPaint file 

- CompuServe Graphics Interchange Format file (GIF) 

- Encapsulated Postscript file (EPS) 

- Hierarchical Data file 

- PIXAR picture file 

- Portable Bitmap file 

- Postscript file 

- SiliconGraphics Image RGB file (SGI) 

- Stardent AVS X image file 

- Sun Rasterfile 

- Tagged image file (TIF) 

- Utah Run-Length-Encodcd image file 

- Wavefront raster image file 

- XI 1 bitmap file 
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The RGB format was chosen to be utilized because it is easily translated into NTSC (National 
Television Standard) and exists as a standard format for both the CCD camera output and the SiliconGraphics 
workstation. 

1. The RGB Format 

The RGB format is a 32 bit digital format to describe a single pixel in terms of its basic color 
components (red, green, and blue). In the C language, a 32 bit format is referred to as a “long integer” data 
type. Eight bits are used to represent each of the intensities for the red, green, and blue color components of 
the pixel. This allows for intensity values to range from 0 to 255 decimal (0 to FF hexadecimal) for each 
component An intensity value of 0 represents zero light intensity for that color component, 255 for full 
illumination. 

The RGB format uses the red, green, and blue colors as the basic components because they are the 
three primary additive colors. An additive color scheme used on television and similar displays combines 
colors from red, green, and blue light sources together to create the rest of the colors in the visual spectrum. 
This is contrasts with the subtractive color scheme implemented with yellow, cyan, and magenta as the three 
primary colors which is used in Film photography for Filtering a white light source to create the other colors. 

The RGBA format only differs from the RGB format in that it has eight bits reserved for the alpha 
component which is used for transparency on graphics workstations. An alpha component with a value of 
zero is completely transparent, whereas a value of 255 is completely opaque. The RGB and RGBA formats 
are both 32 bits long and are basically identical. The RGB format simply ignores the alpha component and 
operates as if all colors are opaque (alpha value set to FF hexadecimal). Figure 7.1 shows the bit placement 
for the RGB/RGBA formats. In order to allow easier translation between these two formats, the alpha 
component is set to FF hexadecimal. To illustrate this format with some examples, the color black is 
represented as FF000000 hexadecimal (alpha = FF; blue, green, red = 00), white is FFFFFFFF, and bright 
kelly green is FF00FF00. 
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Figure 7.1 . The RGB / RGBA formats. RGB format ignores the alpha block. 
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2. Color to Grayscale Conversion 



Thzfindedge implementation of feature extraction requires a grayscale image. However, if a color 

camera is used, it will be necessary to convert the color image to a grayscale image. The RGB format 

describes each pixel in terms of its basic additive primary colors, whereas the overall intensity of the attributes 

would be a grayscale value. To determine the grayscale value, each of the color values must be weighted and 

summed. The weights assigned to each color for representation as a black and white television (grayscale) 

pixel have been standardized by the National Television Systems Committee. The following weights are 

prescribed by the NTSC [HAL89]. 

Red = 0.299 
Green = 0.587 
Blue = 0.114 



Converting a color pixel to a grayscale value is simply assigning the sum of the weighted color 
component values to the grayscale value (equation 7.1). 



RGB grayscale 



[o.299 0.587 0.11 4] 
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G 

B 



(Eq 7.1) 



4. The NPSIMAGE Data Structure 

The RGB format is a well known and widely used format for defining a single color pixel. 
However, formats for describing entire images via RGB pixels can still vary. One such format, a simplified 
model of a standard video image, developed by Mike Zyda at the Naval Postgraduate School, is the 
NPSIMAGE data structure outlined in the image jypes.h file in Appendix A. NPSIMAGE is a C language 
data structure (a record type) to store an RGB, RGB A, or color-mapped image. 

For all video images, the pixels are represented as a single, large, one-dimensional array of values 
starting with the first pixel in one comer of the two-dimensional image and ending at the opposite comer 
(Figure 7.2). All pixels in this array are arranged via a single, linear scan of the video image (i.e. left to right, 
bottom to top). In order to properly display the image from a saved format, it is crucial to have the dimensions 
of the original image saved. The dimensions of the input image are expressed in number of pixels in the 
horizontal and vertical directions. NPSIMAGE has two integer value slots, xsize and ysize, to store these 
dimensions. 
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The image format must subsequently provide an efficient way to access the one-dimensional array 
of pixel values. This is generally done in the C language via a pointer initialized to the first pixel of the one- 
dimensional array. Moving the pointer by altering its offset allows easy access to all pixel values. When using 
the NPSIMAGE data structure, such a pointer will be initialized to the imgdata .bitsptr long integer 
array (Figure 7.3). 




Figure 7.2. Construction of one-dimensional array of pixel values from two-dimensional video image. 
Order of pixel scan is left to right, bottom to top, as used in the NPSIMAGE data structure. 



NPSIMAGE *img; /* a pointer to the image data structure */ 

long *ptr; /* a pointer to pixel values 

(represented as long integers) */ 

ptr = img->imgdata .bitsptr ; 

Figure 7.3. Declaration and initialization of a C language pointer to the RGB values in an NPSIMAGE. 
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VIII. EDGE EXTRACTION RESULTS 



A. E1NDEDGE IMPLEMENTATION 

The implementation of the finciedge algorithm described in Chapter VI and detailed in Appendix B 
provides extracted line-segment features such as in Figure 8 . 1 . The line segments found are from the input 
image of the chair (Figure 3.3). The five parameters for gradient thresholds and line segment properties (C/ 
through C 5 ) were determined by trial and error. The effects of modifying the parameters is seen in later in this 
Chapter. 




Figure 8 . 1 . Lines found from image of chair via findedge implementation. 
Cj = 5,000.000.0: C : = 20.0 degrees; C 3 = 0.1; C 4 = 20 pixels; C 5 = 20.0. 
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B. STATIC METHOD OF GRADIENT ANGLE TESTING 



Results from the static method for determination of “closeness” of pixel gradient angle orientation are 
displayed in Figures 8.2 through 8.4. The input image is that of the chair (Figure 3.3) and the resultant line 
segments can be compared with Figure 8.1. All five parameters are the same as those used in Figure 8.1. 
Figure 8.2 shows the lines found using the first group of static angles divided by 20 degrees (see Figure 6.2) 
and Figure 8.3 shows the lines from the second group of static angles. Figure 8.4 displays all lines found from 
both Figures 8.2 and 8.3. Comparing Figures 8. 1 and 8.4 demonstrates the shortcomings of the static method. 
The line segments are shorter, broken, and subsequently more numerous, all of which are traits not desired 
for the purposes of pattern-matching. 



Figure 8.2. Lines found from image of chair via static gradient angles method (group 1 of Figure 
6.2). Cj = 5,000,000.0; C 2 = 20.0 degrees; C 3 = 0.1: C 4 = 20 pixels: C 5 = 20.0. 
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Figure 8.3. Lines found from image of chair via static gradient angles method (group2 of Figure 
6.2). Cl = 5,000,000.0; C2 = 20.0 degrees; C3 = 0.1; C4 = 20 pixels; C5 = 20.0. 




Figure 8.4. All lines found from image of chair via static gradient angles method 
C, = 5.000,000.0; C 2 = 20.0 degrees; C 3 = 0.1; C 4 = 20 pixels; C 5 = 20.0. 
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C. FASTEDGE IMPLEMENTATION 



The last algorithm described in Chapter VI, the fastedge method produced lines very similar to those 
found by the findedge method. Figure 8.5 shows the resultant line segments via the fastedge implementation. 
The major difference between this and findedge was determination of the gradient magnitude threshold value 
C |. Findedge used color to grayscale conversion for an absolute intensity value where as fast edge only used 
the green color component of the RGB image. 




Figure 8.5. Lines found from image of chair via fastedge implementation. 
C, = 30,000.0; C 2 = 20.0 degrees; C 3 = 0.1; C 4 = 20 pixels; C 5 = 20.0. 
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D. EFFECTS OF EDGE EXTRACTION AFTER “SHRINKING” AN IMAGE 



Condensing an input image to half of its dimensions prior to performing the straight edge feature 
extraction routines produces some noteworthy results. Aside from requiring only a quarter of the 
computational time for edge extraction, the resultant line segments are generally fewer, longer, and seem to 
model the image in simpler terms. To illustrate such differences, Figure 8.6 is resultant line segments (via 
fastedge) of the chair image condensed to half of its length and width (quarter of the original area). Its results 
can be compared with the lines extracted in Figure 8.5. Similarly, Figure 8.7 is a view of a hallway interior 
with its extracted line segments (Figures 8.8 and 8.9) to be compared with the edges extracted from the same 
image after shrinking (Figure 8.10) and edge extraction (Figures 8.11 and 8.12). 




Figure 8.6. Lines found from image of chair after it has been shrunken to half of its 
dimensions and using the fastedge program. Cj = 30,000.0; C 2 = 20.0 degrees; C 3 = 0.1; 
C 4 = 20 pixels; C s - 20.0. 
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Figure 8.7. Image of hallway interior. 




Figure 8.8. Extracted edges of hallway interior. Edge extraction by fastedge method. 
C, = 30.000.0; C 2 = 20.0 degrees; C 3 = 0.1: C 4 = 20 pixels; C 5 = 20.0. 



38 






Figure 8.9. Extracted edges superimposed over image of hallway interior. 
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Figure 8. 10. Shrunken image of hallway interior. 




Figure 8. 1 1. Extracted edges of shrunken hallway interior. Edge extraction by fastedge method. 
C| = 30,000.0; C 2 = 20.0 degrees; C 3 = 0.1; C 4 = 20 pixels; C5 = 20.0. 




Figure 8.12. Extracted edges superimposed over image of hallway interior. 
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E. EFFECTS OF VARYING FEATURE EXTRACTION PARAMETERS 



1. Gradient Magnitude Threshold Cj 

The threshold value Cj is used to determine if the gradient magnitude of a pixel is sufficiently 
large enough to describe an edge. If the value for pixel gradient magnitude, determined by equation 4.1, is 
larger than Cj, then the pixel in the gradient image is set black, otherwise it is set white. In the findedge 
program, the value for C\ was determined to be 5,000,000.0 by testing. Figures 8.9 through 8.11 show the 
associated gradient images for values of Cj equal to 5M, 8M, and 2M, respectively. 
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Figure 8. 10. Gradient image of a chair with magnitude threshold C\ = 8,000,000.0. 




Figure 8.1 1. Gradient image of a chair with magnitude threshold Cj = 2,000,000.0. 
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2. Gradient Angle “Closeness" Angle C 2 



The threshold value Q, the maximum difference between two pixel gradient orientations was 
tested by trial and error to produce more extracted line segments that best represented the straight edges from 
the input image. For the image of the chair (Figure 3.3), C 2 = 17.2 degrees performed best. Larger angles 
resulted in combining multiple straight edges from the image into one extracted line segment, whereas smaller 
angles were too restrictive to produce long line segments. The best Q value for one image is not necessarily 
the best value for all images. Pictures of the hallway interior, used for pattern-matching and pose- 
determination, utilize C 2 = 28.65 degrees (0.5 radians). The images in Figures 8.12 through 8.14 illustrate the 
effects of modifying Q of the chair image via the fastedge implementation. 




Figure 8.12. Lines extracted from image of chair for C 2 = 17.2 degrees (0.3 radians). 
C, = 30,000; C 3 = 0.1; C 4 = 20 pixels; C 5 = 20.0. 
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Figure 8.13. Lines extracted from image of chair for C 2 = 30 degrees. 
C, = 30,000: C 3 = 0. 1 : C 4 = 20 pixels: C 5 = 20.0. 




Figure 8. 14. Lines extracted from image of chair for C 2 = 12 degrees. 
C, = 30.000; C 3 = 0.1: C 4 = 20 pixels: C 5 = 20.0. 
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3. Line Test Parameters Cj, C 4> and C 5 



The three line test parameters identified in Chapter V are utilized for limiting the number of 
extracted line segments by placing restrictions on the ratio of axes lengths (p = d^^d^j^), the number of 
pixels (j moo ), and the major axis length (d^j^), respectively. The lines extracted from the chair image by 
modifying C 3 , C 4 , and C 5 are shown in Figures 8.15 through 8.17. Each Figure can be compared with the 
baseline extracted features in Figure 8.12. 




Figure 8.15. Lines extracted from image of chair for maximum thinness ratio C 3 = 0.05. 
C! = 30,000; C 2 = 17.2 degrees; C 4 = 20 pixels; C 5 = 20.0. 
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Figure 8.16. Lines extracted from image of chair for minimum number of pixels C 4 = 40 pixels. 
Cj = 30,000; C 2 = 17.2 degrees; C 3 = 0.1; C 5 = 20.0. 




Figure 8.17. Lines extracted from image of chair for minimum axis length C 5 = 50.0. 
C\ = 30,000; C 2 = 17.2 degrees; C 3 = 0.1; C 4 = 20 pixels. 
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IX. MATCHING LINE SEGMENTS 



The focus of this Chapter is the development of a matching algorithm that compares the straight edges 
extracted from the input image with the linear features of the environment model. The pattern matching of 
two-dimensional line segments can be divided into two different problems: 

1. matching with the model based upon a known location and orientation of the robot (the pose 
or configuration of the robot), 

2. and matching with the model without knowledge of the robot’s pose. 

The first problem is applied to position correction and vehicular navigation, whereas the second 
problem is that of object identification and viewing aspect determination. The first problem is much more 
constrained than the second and is pursued herein towards the goal of pose determination and vehicle 
navigation. Expecting small dead-reckoning errors to arise in the vehicle’s pose, the matching algorithm must 
exhibit some degree of robustness if it is to make valid corrections. Therefore, the goal of the matching 
process is to find the best match of image lines to model lines. 

A. THE ENVIRONMENT MODEL 

The testing and operating environment for Yamabico- 1 1 is the fifth floor of Spanagel Hall at the Naval 
Postgraduate School. Modeling this environment in a graphical database [STE92] was accomplished in 
conjunction with the above edge extraction methods for the pattern matching task described in Chapter IX. 
The model is a “2-D +” model similar to that described by David Marr [MAR79]. The environment model is 
designed to support requirements for visual navigation, sonar navigation, shortest distance path 
determination, and safe path planning. The floorplan features are expressed in the xy cartesian plane and 
associated heights of hallway features are modelled in the r axis. The notion of free space is aided by defining 
polygons in the xy plane as either floor or ceiling polygons. All database measurements are in inches. All 
support routines for implementation of the model are coded in ANSI C and described fully in [STE92]. 

1. Interfacing the Model Database 

Since the application features of the environment were encoded in the same language as the image 
processing and pattern matching methods, accessing the model is quite simple. The pattern matching routine 
will invoke calls to both the image processing and environment model functions to access the line segment 
primitives utilized in the matching process. The primary call to the model {get_yiew()) requires providing an 
estimated pose (position and orientation information) of the robotic vehicle and the focal length of the camera 
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lens. The function returns a data structure that includes pointers to linked lists of the line segments of the 
model database mapped to a two-dimensional viewing plane that the robot should expect to see given the 
input pose and visibility constraints of occluding edges. The line segments shown in the model’s two- 
dimensional view represent the orthogonal features of the operating environment, primarily the junctures of 
the walls, ceiling, floor, doors, and overhead lights. 

B. BASIC COMPARISON OF LINE SEGMENTS 

Since the environment model provides a two-dimensional view of the known linear features for a given 
pose, the matching algorithm needs only to determine linear matches in two dimensions. From the results seen 
in Chapter VII, the extracted edges from the input image do not cross, overlap, or touch. The extracted linear 
edges are, at best, incomplete segments of the modelled linear features. The matching algorithm must be 
based upon individual matches between the image’s extracted edges and the model’s linear features. When 
comparing two-dimensional line segments, three aspects must be considered: translation, rotation, and scaling 
differences (Figures 9.1 through 9.3). 




Figure 9.1. The translation distance, d , between two lines. 
Can be computed as the median between distances dl and 
dl s perpendicular to the model line. 




Figure 9.2. The rotational difference, a, between two lines. 
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Figure 9.3. The scaling factor, s, between two lines. S is the ratio of the image line 
mapped onto the model line with the model line itself. 



An additional requirement based upon the properties of the extracted edges using least squares fit can 
also be stipulated. That is, the endpoints of the image edges must lie within the endpoints of the modelled line 
segment. This check for endpoint inclusion may be relaxed to allow for vehicle translation errors. This is done 
by keeping the difference that the image edge endpoints lie outside of the endpoints of the modelled line 
segment, divided by the length of the image edge {d ma j or ) y less than or equal to a specified amount ( k in Figure 
9.4). 




Figure 9.4. The inclusion of image endpoints within the model line’s 
endpoints. Image line 2 can meet this requirement if (A/ 1)< k , (0.0 < k < 0.1). 



Another consideration for matching image lines to model lines is that since the extracted image lines 
are often incomplete and broken segments of a modelled feature, many image lines may be matched to one 
model line as shown in Figure 9.5. The only stipulation is that the image lines do not overlap when mapped 
onto the same model line. 
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model line 



3 

Figure 9.5. Many-to-one relationship of image lines to model lines. 

Image lines 1, 2, and 3 may all be matched with the model line. 

These five aspects of two-dimensional line matching could be combined to produce one value to 
describe the confidence for the match of one image line to one model line. Summing individual matches with 
highest confidences provides the most likely matching between image edges and model line segments, yet it 
remains to be determined if it is the correct matching. Beveridge [BEV90] includes an error term for model 
line segments that are omitted from the matching process. Many graph-based matching algorithms outlined 
by Ballard [BAL82] are NP-complete problems. Beveridge’s method defaults to continually permuting 
various matching combinations in order to find the optimum matching. 




C. MATCHING IMPLEMENTATION FOR VERTICAL LINE SEGMENTS 

Assuming that the robot’s camera is in a fixed position and that the robot is on a stable platform, the 
pose (or configuration) of the robot can be described in three degrees of freedom ix 0f y 0 Bo)* With this 
assumption, all vertical lines in the real world will be vertical in any image. Therefore, matching only vertical 
lines should provide a simple solution for the pose determination/correction problem. Since all line segments 
will be vertical, rotational differences between the image lines and model lines may be neglected, and the 
translational distance, d , can be simplified to be the horizontal angle from the midpoint of the image line to 
the model line. 

Considering an image plane M that exists at focal length /from the center of the lens system of the 
robot’s camera, the vertical edges extracted from the image will appear as in Figure 9.6. For a given estimated 
pose (.V0.y0.9fl)* an overview of the robot and the image plane M (Figure 9.6) will be situated in the operating 
environment model as in Figure 9.7. The environment floor is the xy plane and heights of modelled features 
are expressed in the r-axis. Image plane M is parallel with the r-axis. 
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Figure 9.6. The image plane M . The image is centered on image axes u,v with three 
extracted vertical lines at u Jt u 2 , and u 3 . 




Figure 9.7. The robot in the environment coordinate system. The robot’s pose is defined by (x a , y 0 , %). 
The image plane M exists at the focal length (f) from the robot’s camera. Vertical lines extracted from 
the input image in Figure 9.6 (at image u-coordinate positions uj, u 2> and u*) can be described by an 
angle a from the center of the image. 
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A two-dimensional view of the environment model is constructed given the three-dimensional model, 
an assumed pose, and the focal length / (Figure 9.8). The assumed pose is provided by the robot’s dead 
reckoning capabilities from wheel motion and is used knowing that it is an approximation. Therefore, the 
matching problem to evaluate how well the image edges from the image plane M fit onto the model lines in 
the two-dimensional plane M\ For this implementation using only vertical lines, quantifying the match 
between an edge and a model line will be based only upon the translational distance d, if the endpoint 
inclusion requirement is met. 




Figure 9.8. The image plane M\ M ’ is the two-dimensional view of the modelled environment 
for the given pose (*o,yo,%) of the robot (camera). M ' can be thought to exist at a distance equal 
to the focal length,/, of the camera’s lens system. 



The line matching provides connectivity between the extracted image lines and the vertical features of 
the environment model. Thus, the three most significant extracted image lines can be matched with three 
vertical model lines as in Figure 9.9. Vertical image lines uj. u 2 , and uj , from the image plane M, are best 
matched with vertical model features LI , L2, and L3, respectively. This does not ensure that this is the correct 
matching, but it will provide a means to calculate a possible pose for the robot. This method also assumes that 
the three most significant vertical image lines are produced from modelled linear features. 
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Since backtracking may be necessary for recovering from successive attempts in the pose 
determination algorithm, every image edge in M must be able to support a linked list to all of the possible 
matches to the model line segments in M\ Appendix C contains the code for this implementation, titled 
vert match. A data structure named MATCHTYPE in the fide match types Ji (Appendix A) is used for the 
association of an extracted edge with a model line segment. Each image edge data structure has a pointer to 
the head of the list of possible MATCHTYPEs and a separate pointer for the best. The different possible 
matches of an image edge to various model line segments are ordered by horizontal angular difference 
between the location of image edge and model line. The smallest difference, that being the best match, is at 
the head of the match list and all subsequent matches follow. The pose determination/correction algorithm in 
Chapter X utilizes only the three most significant extracted image edges (i.e. the three image lines with the 
greatest values for d ma j or ). Thus the vertmatch implementation creates a list of MATHCTYPEs for the three 
most significant edges. 



y 




Figure 9.9. Modelled vertical features matched with extracted vertical edges from an image. LI, L2, 
and L3 are the labels assigned to the vertical lines matched to the three most significant vertical 
image lines in found in M. Angle y is the orientation of the line from the camera’s position to the 
location of a vertical line. The modelled vertical line should then appear at an angle (0 O - y) from 
the center of plane M\ the 2D view of the model environment (Figure 9.8). 
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X. CORRECTING ROBOT’S POSE 



With the assumption that the robotic vehicle is a stable platform with its camera at a fixed position, 
there are only three degrees of freedom in the model space (x y v, and 0 - corresponding to the variables of the 
robot's pose) that need to be determined. If considering only vertical lines for matching, it is possible to 
determine the camera’s pose from only the three most significant vertical lines in an image, provided that they 
are all produced from features represented in the three-dimensional environment model. 



A. POSE DETERMINATION 



The only known, accurately measured quantities that are available for pose determination are the jrv 
coordinate map locations of the three vertical model lines and the horizontal angular differences between the 
extracted vertical edges u lt u 2 , and u 3 from Figure 9.6. Let us denote two of these angles, the angle between 
the leftmost (uj) and middle (u 2 ) image lines and the angle between the middle ( u 2 ) and rightmost ( u 2 ) image 
lines as a and p. respectively. If the best matches for u } . u 2> and uj are used (vertical model lines LI , L2, and 
L3), angle a is the expected viewing angle between LI and L2, and p is the expected viewing angle between 
L2 and L3. There is exactly one location that will have viewing angles a and P between model lines LI, L2, 
and L3. If the distance between two vertical model features is thought of as the side of a triangle (a) and the 
associated viewing angle (a) as the opposite angle of the triangle, an infinite number of possible locations for 
the viewing angle exist on a circle that circumscribes all possible triangles (Figure 10. 1). For two triangles, 
one with side a and opposite angle a, and the other with side b and angle p, two circles of possible viewing 
locations can be constructed as in Figure 10.2. Sides a and b are therefore chords of the two circles. There are 
two intersections of the circles, provided that the circles are not identical. One intersection is at the location 
of vertical model line L2 and the other is the possible position of the camera Qt poss , y poss ). The camera must 
be at this position for this line matching, given the viewing angles a and p between the three major vertical 
image edges in M . 




Figure 10.1. Triangles with side a and opposite angle a, circumscribed by a circle. 
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►— (Xpos&yposs) 

possible correct robot position 



Figure 10.2. Geometry for pose determination. Given measured locations of vertical features 
in the environment model (LI, L2, and L3), circle chords a and b can be calculated. Angles 
a and p are measured from the image plane and are defined by the angular difference between 
the vertical extracted edge features from the image. With chords a and b and angles a and p, two 
circles can be constructed such that their intersections will exist at the position of model line L2 
t'L2v v L2) and a possible robot position y poss ). 



Once a position has been determined, calculation of the camera’s orientation in the environment is 
simple. Since the locations of the vertical modelled features are known in the environment map (xy, y L i), the 
angular orientation, y, from the possible position of the camera (jc^, y poM ) is atan2(jt Li - Xponv yu m Jposs)- 
The angular location of these vertical model lines from the vertical centerline axis in the two-dimensional 
plane M' is 0 C - y. Averaging the differences between angles of the model lines from the center of the plane 
M' (9 C - \\f) and the angles of the image lines from the center of the image plane M provides a rotational 
correction. The correction is added to the orientation of the input (dead-reckoning) pose, 9 0 , to get the 
possible pose orientation, 9 poss . The pose for the camera based upon the given matching is therefore 



determined (x 
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B. POSE VERIFICATION 



The next best match combination will be performed by selecting the next match with the least 
horizontal angle difference between the vertical image edge and model line segment out of all three image 
line match lists. For this new combination of three matches, another possible pose is calculated by the above 
method. This is done for the n best match combinations. An exception to this rule is to skip matches where 
two or more image lines are matched to the model line segment. This method will provide, at most, n possible 
poses from the n match combinations. 

Since a unique pose can be determined from any given matching combination for the three most 
significant image edges, the determination of the correct pose of the robot must rely upon a means to verify 
the pose and the matching combination. This topic is described by Heller and Stenstrom [HEL89]. For a given 
possible pose (a^,, y pOM , 0^,,), locations for all the model line segments can be calculated if they were to 
be viewed in the image plane M. Comparing all of the extracted image lines to see how many fit the 
repositioned model lines provides a means to evaluate the possible pose. However, since many image lines 
are not produced from modelled features using the number of good image to model line fittings alone can not 
guarantee the correct pose. 

If some credibility is expected of the estimated pose for which the two-dimensional view of the model 
(Af’)was constructed from, then calculated possible poses should be close to the estimated pose. Thus, an 
evaluation ( eral poss ) combining the number of image edge to model line segment fittings (m poss ) with the 
distance between the possible pose and the estimated pose ( d poss ) will be of the form 



, mposs 

rr = — ~ — . 
p OS S sJ 

poss 



(Eq 10.1) 



The possible pose with the greatest value for {exal poss ) will then be considered the corrected pose. 
Results of this method are shown in Chapter XI. 
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XI. PATTERN MATCHING AND POSE DETERMINATION RESULTS 



The results from three trials are shown in this chapter. The actual camera pose is measured from a 
reference point in the operating environment The estimated pose for developing the two-dimensional view 
of the model environment was encoded in the program vertmatch to simulate dead-reckoning errors. Each 
trial has a series of four figures. The first figure is the input image with the extracted vertical edges. The 
second figure is a wire frame two-dimensional view of the environment model based upon the estimated pose. 
The third figure is the wire-frame model superimposed over the input image. The fourth figure is a wire-frame 
model based upon the corrected pose superimposed over the image. The inputs to the vertmatch routine are 
the image, the environment model, and the estimated pose. Output from the vertmatch routine listing the 
matches and possible poses is shown between the third and fourth figures of each trial. 



A. TRIAL 1 




Figure 11.1. Trial 1, input image with extracted edges. 

Actual camera pose: Xq = 60.0 inches, y 0 = 366.0 inches, 6 0 = 253.0°. 
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Figure 1 1.2. Trial 1, 2D wire-frame view of model based upon estimated pose. 
Estimated pose: xq = 60.0 inches, yo = 366.0 inches, 9 q = 250.0°. 




Figure 1 1.3. Trial 1, 2D wire-frame view superimposed over input image. 
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Script started on Thu Mar 19 12:12:18 1992 
turing 1% vertmatch 060366253 . pic 

vertmatch> 060366253 . pic xsize= 646 ysize= 486 pixels^ 313956 
lines found in image written to: 'lines. text' 

Number lines found in 060366253 . pic = 17 

DR (input) pose: x = 60.00, y = 366.00, theta = 250.00(-1.92 rads) 

Nr Vertical Model Lines = 20 

Determine 3 major image lines for matching: 

1) line 13, angle from center = -0.1819 

2) line 10, angle from center = 0.0980 

3) line 8, angle from center = -0.0814 

Image line matchlists to Model line NAME (angle difference): 

left (edge 10) > 

N (0 .0 526) M (-0 . 0 634 ) L(-0.0675) I(-0.0904) J(-0.0904) H(-0.1279) 

G (-0 . 132 1 ) F (-0 . 1737) E(-0.2176) D(-0.2224) C(-0.2288) A(-0.2300) 

middle (edge 8) > 

F (0 . 0057 ) E (-0 . 0382 ) D(-0.0430) G(0.0473) C(-0.0494 ) A(-0.0506) 

H (0 . 0515 ) J (0 . 08 90 ) 1(0.0890) L(0.1119) M(0.1160) N(0.2320) 

right (edge 13) > 

A ( 0 . 0 4 98 ) C (0 . 05 11 ) D(0.0575) E(0.0623) F(0.1062) G(0.1478) 

H ( 0 . 1520 ) J ( 0 . 18 95 ) N(0.3325) 

Pose determination: 

[ N, F, A] x= 55.42 y= 513.24 T= 147.3124 R= -0.3730 nr verify matches= 5 
*** BEST POSE MODIFIED *** 

[ N, E, A] x= 276.04 y= 256.66 T= 242.1306 R= -0.6891 nr verify matches= 0 
[ N, D, A] NO position: chord length < MIN_CHORD_LENGTH 

[ N, G, A] x= 51.84 y= 381.92 T= 17.8955 R= 0.0048 nr verify matches= 6 

*** BEST POSE MODIFIED *** 

[ N, C, A] NO position: chord length < MIN_CHORD_LENGTH 
[ N, A, A] NO position: 2 IMG_LINEs matched to same model LINE 
[ N, A, C] NO position: chord length < MIN_CHORD_LENGTH 

[ N, H, C] x— 58.49 y= 368.04 T= 2.5417 R= 0.0464 nr verify matches= 7 

*** BEST POSE MODIFIED *** 

[ N, H, D] x= 58.48 y= 379.09 T= 13.1814 R= 0.0087 nr verify matches= 7 

[ N, H, E] x= 86.10 y= 459.31 T= 96.8872 R= -0.2843 nr verify matches= 6 

[ M, H, E] x= 307.54 y= 268.49 T= 266.0523 R= -1.1971 nr verify matches= 0 

[ L, H, E] x= 307.41 y= 268.42 T= 265.9568 R= -1.1694 nr verify matches^ 0 

[ L, J, E] NO position: chord length < MIN_CHORD_LENGTH 

[ L, I, E] NO position: chord length < MIN_CHORD_LENGTH 

[I, I, E] NO position: 2 IMG_LINEs matched to same model LINE 
[ J, I, E] NO position: chord length < MIN_CHORD_LENGTH 

[ J, I, F] NO position: chord length < MIN_CHORD_LENGTH 

[ J, L, F] NO position: chord length < MIN_CHORD_LENGTH 

[ J, M, F] NO position: chord length < MIN_CHORD_LENGTH 

( H, M, F] x= 276.86 y= 247.81 T= 246.9754 R= 0.8255 nr verify matches= 2 

[ G, M, F] x= 270.77 y= 246.45 T= 242.3155 R= 0.8120 nr verify matches= 2 

[ G, M, G] NO position: 2 IMG_LINEs matched to same model LINE 
[ G, M, H] x= 307.93 y= 268.25 T= 266.5063 R= -1.4201 nr verify matches= 0 

[ F, M, H] x= 340.40 y= 274.86 T= 294.8414 R= -1.5850 nr verify matches= 0 
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[ A, N, J] x= 337.19 y= 307.83 T= 283.2245 R= -0.5412 nr verify matches= 0 
[ A, N, N] NO position: 2 IMG_LINEs matched to same model LINE 

Corrected pose: x= 58.49, y= 368.04, theta= 252 . 66 (-1 . 8735 rads) 
turing 2% exit 
turing 3% 

script done on Thu Mar 19 12:13:35 1992 




Figure 1 1.4. Trial 1, corrected 2D wire-frame view superimposed over input image. 
Corrected pose: xq = 58.49 inches, y 0 = 368.04 inches, 0 q = 252.66°. 

Translational error from actual pose = 2.54 inches, rotational error = 0.34°. 
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B 



TRIAL 2 




Figure 1 1.5. Trial 2, input image with extracted edges. 

Actual camera pose: x 0 = 59.0 inches, y 0 = 366.0 inches. Go = 249.0°. 
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Figure 1 1 .6. Trial 2, 2D wire-frame view of model based upon estimated pose. 
Estimated pose: x 0 = 60.0 inches, y 0 = 366.0 inches, 0 O = 250.0°. 




Figure 1 1 .7. Trial 2, 2D wire-frame view superimposed over input image. 
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Figure 1 1.8. Trial 2, corrected 2D wire-frame view' superimposed over input image. 
Corrected pose: x 0 = 58.73 inches, y 0 = 366.45 inches, 0 O = 249.02°. 

Translational error from actual pose = 1.35 inches, rotational error = 0.02°. 
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C. TRIAL 3 




Figure 1 1 .9. Trial 3, input image with extracted edges. 

Actual camera pose: x 0 = 43.0 inches, y 0 = 277.0 inches, 0 O = 356.0°. 




Estimated pose: xq = 48.0 inches, y 0 = 277.0 inches, 0 O = 000.0°. 
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Figure 11.11. Trial 3, 2D wire-frame view superimposed over input image. 
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Figure 11.12. Trial 3, corrected 2D wire-frame view superimposed over input image. 
Corrected pose: x 0 = 46.55 inches, >'q = 260.39 inches, 9o = 356.42°. 

Translational error from actual pose = 16.67 inches, rotational error = 0.42°. 



68 





XIL CONCLUSIONS 



A. FEATURE EXTRACTION 

The edge extraction implementation fastedge , provides linear features that, unlike the Hough 
transform, have good endpoint information. The implementation requires approximately 15 seconds to 
extract the linear edges from a 486 x 646 pixel image (313,956 pixels total) on the Personal Iris (35 MHz 
clock cycle). For a picture that has been “shrunken” to half of its width and height, the process requires only 
3 seconds. 

Selecting the appropriate constants C ; through C 5 is important. The effects of modifying these 
parameters, outlined in Chapter VII, provide optimum values for use in a specific environment. The task of 
determining optimum values for a dynamic environment could be automated; however, it would require an 
extra scan through the image or else added hardware (e.g. a lightmeter). 

The version implementing two static sets of angles used for the description of pixel gradient 
orientations could provide a very fast hardware implementation. Although it did not perform as well as 
averaging gradient orientations, the speedup advantages gained from implementing this method warrant more 
consideration and experimentation. 

B. PATTERN MATCHING 

Pattern matching is the “weak link” in the xertmatch implementation. Although the method is quick, 
it is crude. It was chosen as a means to provide a simple basis for the pose determination problem, therefore 
opting for the easier problem of matching only two-dimensional vertical line segments with the assumption 
that the three most significant edges extracted from the image are products of vertical modelled features. 
Many line matching implementations are graph based and therefore rely upon links between the extracted line 
segments that can only be inferred, but not accurately determined. Graph based matching algorithms are often 
NP complete thus leading to alternative methods like Beveridge’s randomly permuting matches to find the 
optimum match. 

C. POSE DETERMINATION 

The pose determination algorithm was based upon simple vertical line matching in a known orthogonal 
environment and the geometry for its foundation allows a unique pose to be calculated simply and quickly. It 
was found to provide good estimates from three visual lines of bearings. Small errors between the actual and 
corrected poses in the three trials in Chapter XI could be from the algorithm, or from hardware floating-point 
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constraints involved with the trigonometric functions, or (most likely) from data measurement error from the 
tape measure. 

The pose determination test trials were conducted at two different primary locations, viewing the 
elevator alcove and viewing the long axis of the hallway. Trials 1 and 2 in Chapter XI were conducted with 
the environment features of the elevator alcove approximately 20 feet from the camera which provided only 
20 vertical model line segments. With relatively few model line segments for the three major image edges to 
be compared with, the matching was easy to check. Trial 3 involved much greater distances and over 100 
vertical model line segments. The corrected pose could be more accurate if more iterations for possible 
matches and poses are conducted. However, the actual pose can be determined if all match combinations are 
tested exhaustively. Implementation of backtracking with some good heuristics about the matching process 
would be beneficial here. The long aspect of the hallway for pose determination is more difficult in terms of 
distances and possible matching combinations; however, other features (edges from baseboards and overhead 
lights) may be utilized to enhance the performance. Thus, a more general and more robust line matching 
implementation is required. 

D. FOLLOW-ON WORK FOR YAMABICO-11 

The methods developed in this thesis only have been directed towards the first objective of visual 
navigation in a known environment and has not dealt with the greater complexities of object recognition in 
an unknown environment. There remains much work to be done for continuing the implementation of visual 
navigation capabilities on Yamabico-1 1. Some of these areas are: 

Coding the methods in this thesis in 68020 microprocessor assembly language. 

Developing a vision based support programming language similar to the Mobile Motion 
Language. 

Implementing the environment model support routines in hardware. 

Developing a multiprocessor architecture to handle image edge extraction, two-dimensional 
environment view construction, line matching, and pose determination in parallel. 

Incorporating sensor fusion of sonar data with the vision navigation routines to check accuracy. 

Develop and implement a fast, robust line matching method for all lines. 

Develop a method to positively determine if an edge is from a modelled linear feature. 

Develop object-based recognition methods to handle known and unknown objects. 

Develop an vision based algorithm that supports a reflexive behavior for obstacle avoidance. 

Develop methods to track moving obstacles. 
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APPENDIX A - DATA TYPES 



A. FUNCTION: ATAN2 



I************************************************************************ 

/* file: atan2.c 
/* 

/* Provide a definition of the atan2() function. 

/* 

^******************************3 Ic***:*************************************^ 

#define PI 3.141592653589793 



double atan2(y,x) 
double x, y; 

if (x > 0.0) return (arctan(y/x)); 

else if ((x < 0.0) && (y > 0.0)) return (arctan(y/x) + PI); 

else if ((x < 0.0) && (y < 0.0)) return (arctan(y/x) - PI); 

else if ((x < 0.0) && (y == 0.0)) return (PI); 

else if ((x == 0.0) && (y > 0.0)) return (PI/2.0); 

else if ((x == 0.0) && (y < 0.0)) return (-PI/2.0); 

else return (0.0); 

} 
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B. DATA TYPE HEADER FILE: IMAGE TYPES. H 



/* file: imagc_iypcs.h 
/* 

/* This file holds the structure definitions for the different 
/* types of in-memory images. 

/* Types defined: CMAPIMAGE 
/* NPSIMAGE 

#define RGBA 1 /* RGBA 24 bit images (alpha is Oxff filled) */ 

#define CMAPPED 2 /* color mapped images */ 

#define RGB A WITH ALPHA 3 /* RGBA 32 bit images where alpha is read/saved 
in the image files. */ 

/* define a structure type for color mapped images */ 

struct cmapimage 

{ 

short *bitsptr; /* the bits lor the short images */ 
long nentries; /* the total number of entries in the color map */ 
short *rcds; /* ptr to the red entries of the color map */ 
short * greens; F ptr to the green entries in the color map */ 
short *blues; /* ptr to the blue entries in the color map */ 

long emapoffset; F color map offset, i.e. the first color we will us in the color map */ 

}; 

typedcf struct cmapimage CMAPIMAGE: /* define a CMAPIMAGE type */ 



/* define a union so that the top level image structure’s last pointer can point to several different 
kinds of images. x / 



union imaecdptr 
{ 



long *bitsptr; /* long images need no more data than a ptr to the bits. */ 

CMAPIMAGE *cmapptr; /* a color mapped image must have the bits and a color map so 

we need a complete structure. */ 



/* define the top level structure for the image */ 

struct imaee 

( 

long type; F image t>pc */ 

long xsize; /* xsi/e of the image */ 

long vsi/e; F >si/.e of the image */ 

char *namc; F ptr to string naming the image */ 

union imagedptr imgdata; F ptrs to data for this type of image */ 

}; 

typedcf struct image NPSIMAGE; F define an NPSIMAGE t>pe */ 
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C. DATA TYPE HEADER FILE: MATCH TYPES.H 



I************************************************************************ 

/* file: match_types.h 

r 

/* This file holds the structure definitions for image edges and 
/* match types to model lines. 

r 

/* Types defined: EDGE 
/* POINT 

/* POSE 

/* MATCHTYPE 

/* IMGJLINE 

/********* ********* 4 ************* ********************************* ******* 



typedef struct edge_region_type 

{ 

int active; /* boolean if past region is appended to a present region */ 
long first_pixel; /* first and last pixels added to region */ 
long last_pixel; 

long xmin; /* min & max pixels for previous row */ 
long xmax; 

double avg_phi; /* for use with dynamic averaging of gradient orientation, phi */ 
double sum_phi; 

/* Least Squares Fit momments: */ 
long mOO; /* Number of pixels */ 

double mlO; /* Sum x */ 

double mOl; /* Sum y */ 

double mil; /* Sum x*y */ 

double m20; /* Sum x^x */ 

double m02; /* Sum y*y */ 

struct edge_region_type *next; /* ptr to the next EDGE */ 

} EDGE; 



struct point_type 

double x,y; /* x,y coordinates of the pixel endpoints */ 

}; 

typedef struct point_type POINT; 
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struct posc_type 

{ 

float x, y, theta; 

}; 

typedef struct pose_tvpc POSE; 



typedef struct match_typc 

{ 

LINE * line; /* ptr to LINE type (from Jim Stein’s “graphics.c”) */ 

double angle_view_diff; /* angular difference between image and model lines in the image */ 

float conf; /* confidence value for the match */ 

float dist; /* distance between the *match LINE and IMG JJNE */ 

float scale; /* ratio lMG_LlNE->dmajor/MODEL_LINE->lcngth */ 

struct match_type *next: 

} MATCHTYPE: 



typedef struct line_type 

( 

char namc[3J; 

POINT pi , p2; /* the 2 endpoints for the line */ 

struct line_type ’“next; /* ptr to the next 1MG_LINE in the image */ 

/* Least Squares Fit momments: - */ 

long mOO; /* Number of pixels */ 

double mlO; /* Sum x */ 

double mOl; /* Sum y */ 

double mil; /* Sum \*y */ 

double m2(); F Sum x*x */ 

double m()2; /* Sum y*y */ 

double phi; /* Calculated normal orientation of IMGJJNE */ 
double dmajor; F Length of major axis of equivalent ellipse */ 
double dminor; /* Length of minor axis of equivallent ellipse */ 
double rho; F Ratio dminor/dmajor */ 
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/* Pattern Matching Information: */ 

double angle_to_image__ccnter; 

MATCHTYPE *matchlist; /* List of matches to LINE types */ 
MATCHTYPE *pm; /* present match being considered */ 

} IMG.LINE; 
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APPENDIX B - EDGE EXTRACTION ROUTINES 



A. IMPLEMENTATION: FINDEDGE.C 



/* FILENAME: findedge.c 
/* AUTHOR: Kevin Peterson 
/* DATE: 05 December 1991 
/* 

/^DESCRIPTION: An image gradient program incorporating edge-finding. 
/* Displays edge-gradient image and associated lines. 

/* 

/* This application is designed for use on a Silicon- 
/* Graphics Iris (r) workstation utilizing a .sgi 
/* or similar rgb formatted image. 

/* RGB values are of type LONG in the form AABBGGRR w here: 

/* AA - alpha value, 0-255 
/* BB - blue component, 0-255 
/* GG - green component, 0-255 
/* RR - red component, 0-255 
/* 

/* SiliconGraphics graphics library functions used within 
/* the display _bw_and_gradient_images() routine: 

/* qdeviceO, winset(), c31'0, move20, draw2(), 

/* sw ; apbuffers(), rcshapevicwport(), w'inclose(), 

/* and lrectw rite(). 

/* 

/* NPSIMAGE function routines borrowed courtesy of M.Zyda: 

/* rcad_sgi_rgbimageO, get_empty_rgba_npsimage(), 

/* gct_empty_rgb_npsimauc(), and rgbalong_to_bwlong(). 

/* 

/* Least Squares Fit method for line-finding from 
/* "Sonar Data Inierpretaion for Autonomous Mobile Robots” 

/* by Y.Kanayama, T. Noguchi, & B. Hartman, 1990. 



#include <gl.h> /* SiliconGraphics (r) graphic library */ 

#include <gl/image.h> /* SGI image structure library' */ 

#include <device.h> /* Machine-dependent device library for keys and mouse-buttons */ 
#include <stdio.h> /* C standard i/o library */ 

#include <math.h> /* C math library for aian2() */ 



include “imagc_typcs.h” /* Type definitions for NPSIMAGE, etc. */ 
#include “edge_lypes.h” /* Type definitions for EDGE, LINE, etc */ 



#include "npsimagesupport.h” /* Some NPSIMAGE functions */ 
#include “edgesupport.h” /* EDGE and IMGJJNE building functions */ 
#include “displaysupport.h” /* Graphics display functions */ 



#define THRESHOLD 5000000.0/* for gradient magnitude threshold test */ 
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main(argc, argv) 
int argc; 
char *argv[]; 

{ 

NPSIMAGE *imgl, /* input file_name.rgb color image */ 

*img2, /* black&white image */ 

*img3; /* gradient image */ 

/* pointers to RGBA longs (“bitsptr”s) of respective NPSIMAGEs */ 
long *ptrl, *ptr2, *ptr3; 

double dx, dy, Th = THRESHOLD*THRESHOLD; 
register int i = 0, /* counter for pixels in input image */ 
z = 0; /* counter for pixels in gradient image */ 

EDGE *reg; 

if(argc != 2) fatal( 44 usage: findedge filename\n”); 

/* Read in input rgb image */ 
imgl = read_sgi_rgbimage(argvfl]); 
if(imgl —(NPSIMAGE *) NULL) 

{ 

fatalC 4 File %s is a NULL image.\n”,imgl->name); 

} 

Xdim = imgl->xsize; /* else set global Xdim and ptrl */ 
ptrl = imgl->imgdata.bitsptr; 

printf(“findedge:> %s xsize= %d ysize= %d pixels= %cfsn”, 

imgl->name,imgl->xsize,imgl->ysize, (imgl ->xsize* imgl ->ysize)); 



/* Declare new NPSIMAGEs */ 

if((imgl->type == RGB AWITH ALPHA) II (imgl->type == RGBA)) 

{ 

img2 = get_empty_rgba_npsimage(Xdim,imgl->ysize,imgl->name); 

img3 = gei_empiy_rgba_npsimage((Xdim-2),imgl->ysize-2/Tindedge”); 

else 

{ 

fatal( 44 Unknown or c-mapped image type: %d.W\imgl->type); 

) 

ptr2 = img2->imgdata.bitsptr; 
ptr3 = img3->imgdata.bitsptr; 

/* The scan of an RGB image is from the bottom row -> up, 
traversing the rows left to right. */ 

/* In order for the Sobel operator to be calculated for a specific pixel, 
all eight surronding pixels must have an absolute (black & white) 
light intensity calculated. Function rgbalong_to_b\vk>ng() performs 
this task. */ 

/* Due to the nature of the Sobel operator, the pixels in the top and 
bottom rows as well as pixels in the leftmost and rightmost columns 
of the input image will not be calculated. In order to start cal- 
culating the Sobel operators, the first 2 rows of the input image 
must be converted to black & white light intensity values. */ 
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/* Calculate bw values for first 2 rows of input image. */ 
for(i=0; i<(2*Xdim)+2; ++i) 

{ 

rgbalong_to„bwlong(ptrl[i],&ptr2[i]); 

} 



for(i = Xdim + 1; i < (Xdim*(imgl->ysize-l))-l; ++i) 

{ 

/* Convert color(imgl) to b/w(img2) for pixel on next row up and one 
pixel over to the right so that all eight neighbors of pixel i 
have black & white light intensities. */ 

rgbalong_to_bwlong(ptrl[i+Xdim+l] > &ptr2[i+Xdim+l]); 

/* Ensure pixel i is not in leftmost ro rightmost column */ 
if((i%Xdim != 0) && (i%Xdim != Xdim-1)) 

{ 

/* Calculate dx,dy via Sobel operator for pixel i. */ 

dx = (-ptr2[i+Xdim-l] + ptr2[i+Xdim+l] 

-(2 * ptr2[i-l])+ (2 * ptr2[i+l]) 

-ptr2[i-Xdim-l] + ptr2[i-Xdim+l]); 

dv = ( ptr2fi+Xdim-l] + (2*ptr2[i+Xdim]) + ptr2[i+Xdim+l] 
-ptr2[i-Xdim-l] - (2*ptr2[i-Xdim]) - ptr2[i-Xdim+l]); 



if((dx*dx)-r-(dy*dy) > Thj 

1 

pixel_membcrship(z > atan2(dy,dx)); 

sct_pixel_black(&ptr3[z]); 



else 

( 

set_pixel_w hite(&ptr3[z]); 



++/.; r Increment the pixel counter z for the gradient imaec. */ 

} 

/* If pixel i is in the leftmost column, do check_active_edges(). */ 
else il(i%Xdim == 0) 

( 

chcck_aetivc_cdges(); 

} 

} /* endfor i */ 

/* Check remaining EDGEs for lines: */ 
reg = Passed ucjistjicad; 
while(reg != NULL) 

( 

line_test(reg); 
reg = rcg->next: 



/* Write the lines list to file "lines. text”. */ 
write_all_lines(argv[l] > iing3->xsize > img3->ysize); 
printf(“Number lines found in %s = %d\n”, argv[l]JLinecount); 

/* Display the black&w'hite and gradient images on the screen. */ 
display_bw'_and_£radicnt_images(img2,img3,Line_listJiead); 

prinif(“findedge %s...done.\n’\argv[l]); 

} 
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B. IMPLEMENTATION: FASTEDGE.C 



j********** *********************************************************** j 

/* FILENAME: fastedge.c 
/* AUTHOR: Kevin Peterson 
/* DATE: 06 January 1992 

/* DESCRIPTION: An image gradient program incorporating edge-finding. 

/* Displays edge-gradient image and associated lines. 

/* 

/* This application is designed for use on a Silicon- 
/* Graphics Iris (r) workstation utilizing a .sgi 
/* or similar rgb formatted image. 

/* RGB values are of type LONG in the form AABBGGRR where: 

/* AA - alpha value, 0-255 
/* BB - blue component, 0-255 
/* GG - green component, 0-255 
/* RR - red component, 0-255 

/* SiliconGraphics graphics library functions used within the display_bw_and_gradient_imagesO 
/* routine: qdeviccQ, winset(), c3f(), move2(), draw20, swapbuffersQ, reshapeviewportO, wincloseO, 
/* and lrectwriteQ. 

/* NPSIMAGE function routines borrowed courtesy of M.Zyda: read_sgi_rgbimage(), 

/* get_empty_rgba_npsimage(), get_empty_rgb_npsimageO, and rgbalong_to_bwlong(). 

/* Least Squares Fit method for line-finding from “Sonar Data Interpretaion for Autonomous Mobile 
/* Robots” by Y.Kanayama, T.Noguchi, & B. Hartman, 1990. 

^** ******************************************************************** ^ 

#include <gl.h> /* SiliconGraphics (r) graphic library */ 

#include <gl/image.h>/* SGI image structure library */ 

#include <dcvice.h> /* Machine-dependent device library for keys and mouse-buttons */ 

#include <stdio.h> /* C standard i/o library */ 

#include <math.h> /* C math library for a tan 20 */ 

#include “image_types.h” / a|e Type definitions for NPSIMAGE, etc. */ 

#include “edge_types.h” /* Type definitions for EDGE, LINE, etc */ 

#include “npsimagesupport.h” /* Some NPSIMAGE functions */ 

#include “edgesupport.h” /* EDGE and IMG_LINE building functions */ 

#include “displaysupport.h” /* Graphics display functions */ 

main(argc, argv) 
int arge; 
char *argv[]; 

{ 

NPSIMAGE *iing; /* input file_name.rgb color image */ 
long *ptr; /* pointer to bitsptr of NPSIMAGE */ 
long grmask = OxOOOOffOO; 
double dx, dy, Th = THRESHOLD*THRESHOLD; 
register int i = 0, /* counter for pixels in input image */ 

z = 0; /* counter for pixels in gradient image */ 

EDGE *reg; 

if(argc != 2) fatal(“usage: fastedge filename\n”); 

/* Read in input rgb image */ 
img = read_sgi_rgbimage(argv[l]); 

if(img ==(NPSIMAGE *) NULL) fatal(“File %s is a NULL image.Vi”,img->name); 
printf(“fastedge > %s xsize= %d ysize= %d pixels= %d\n”, 
img->name,img->xsize,img->ysize, (img->xsize*img->ysize)); 

Line_list_head = fastlines(img); 

display_line_image(img,Line_list_head); 
printf(“fastedgc %s...donc.Vi”,argv[l]); 

} 
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C. IMPLEMENTATION: YERTEDGE.C 






/* FILENAME: vcriedge.c 
/* AUTHOR: Kevin Peterson 
/* DATE: 30 January 1992 



r 

/* DESCRIPTION: Same a fastedge.c but only returns vertical edges from image. 
/* 



#include <gl.h> /* SiliconGraphics (r) graphic library */ 

#include <gl/image.h> /* SGI image structure library */ 

^include <dcvice.h> /* Machine-dependent device library */ 

/* for keys and mouse-buttons */ 

#include <stdio.h> /* C standard i/o library */ 

#includc <math.h> /* C math library for atan2() */ 

#include “imagc_typcs.h” /* Type definitions for NPSIMAGE, etc. */ 
^include “cdgc_t>pcs.lT /* Type definitions for EDGE, LINE, etc */ 

#includc “npsimagesupport.h” /* Some NPSIMAGE functions */ 

#includc “edgesupport.h” /* EDGE and IMG_LINE building functions */ 
#inelude “vertsupport.h” /* Vertical EDGE and IMG_LINE supplement */ 
#includc “displaysupport.ir /* Graphics display functions */ 



main(argc, argvj 
int arge; 
char *argv[); 

{ 

NPSIMAGE *img; /* input lile_namc.rgb color image */ 

if(argc != 2) latal(/‘usagc: \ertedge filenamcNn”); 

r Read in input rgb image */ 
img = read_sgi_rgbimage(argv[l]); 

if(img ==(NPS1MAGE *) NULL) fatal(“File %s is a NULL imagc.\n’\img->name); 
prinif(“vcrtcdgc> %s xsize= Vcd ysize= % d pixels= %d\n”, 
img->name,img->xsize,img->ysize, (img->xsize*img->ysizc)); 

Line_list_head = vertlmes(img); 

display_lincJmage(img,LineJist_head); 

printf(“vcriedgc #s...done.VT,argv[l]); 
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D. FILE: NPSIMAG ESUPPORT.H 



/* FILENAME: npsimagesupport.h 
/* AUTHOR: Kevin Peierson 
/* DATE: 29 January 1992 
/* 

/^DESCRIPTION: Collection of basic npsimage functions. 



/* 

/* NPSIMAGE *get_empty_rgb_npsimage (long xsize, long ysize, char name[]) 
/* NPSIMAGE *get_empty_rgba_npsimage (long xsize, long ysize, char name[]) 
/* NPSIMAGE *rcad_sgi_rgbimage (char filename[]) 

/* void write_sgi_rgbimage (char filename[], NPSIMAGE *img) 

/* 

/* V 



/* NPSIMAGE *get_empiy_rgb_npsimage (long xsize, long ysize, char name[]) 
/* 

/* The following function reads in an SGI RGB image as an NPSIMAGE. 

/* - coutesy of M. Zyda, Naval Postgraduate School 
/* V 

NPS I M A G E * g c t_cm pt > _rg b_n ps i m age (x s i ze ,y s i ze ,n am e) 

long xsize, ysize; /* the size the image should be in pixels */ 

char name!]: /* name to attach to the image */ 



NPSIMAGE ^img; ptr to an NPSIMAGE */ 



/* allocate an NPSIMAGE header */ 

img = (NPSIMAGE *jinaIloc(sizeof(NPSlM AGE)); 

/* set the type of NPSIMAGE to RGBA */ 
img->typc = RGBA; 

/* record the widths and height */ 
img->xsize = xsize: 
img->ysizc = ysize; 

/* allocate space for the name of the image */ 
img->name = (char *)malIoc(strIen(name)+l); 

/* copy the name into the string allocated */ 
strcpy(img->name,name); 

/* allocate the memory for the bitmap of the image */ 
img->imgdata.bitsptr = 

(long *)maIloc(sizeof(Iong) * img->xsize * img->ysize); 

/* return a ptr to this empty image */ 
return(img): 

} 
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r */ 

/* NPSIMAGE *gct_cmpiy_rgba_npsimage (long xsizc, long ysize, char name[]) 
/* 

/* The following function reads in an SGI RGB image as an NPSIMAGE. 

/* - couiesy of M. Zyda, Naval Postgraduate School 

* j 

NPSIMAGE *get_cmpty_rgba_npsimage(xsize, ysize, name) 
long xsize, ysize; /* the size the image should be in pixels */ 
char namc[j; /* name to attach to the image */ 

( 

NPSIMAGE *img; /* ptr to an NPSIMAGE */ 



/* allocate an NPSIMAGE header */ 

img = (NPSIMAGE *)malloc(sizcof(NPSIMAGE)); 

/* set the type of NPSIMAGE to RGBA */ 
img->iypc = RGBAW1TH ALPHA; 

/* record the widths and height */ 
i mg' -> xsize = xsizc: 
img->ysizc = ysize: 



/* allocate space for the name of the image */ 
img->namc = (char *)malloc(strlcn(name)+l); 

/* copy the name into the string allocated */ 
strcpy(img->name,name): 



r allocate the memor\ for the bitmap of the image */ 

img->imgdata.bihptr = (long ‘ Jmalloc(sizeof(long) * img->xsizc * img->ysizc); 

/* return a ptr to this cmpt\ image */ 
rciurn(img): 
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/* --*/ 

/* NPSIMAGE *read_sgi_rgbimage (char filename!]) 

/* 

/* The following function reads in an SGI RGB image as an NPSIMAGE. 
/* - coutesy of M. Zyda, Naval Postgraduate School 

y* * j 



NPSIMAGE *rcad_sgi_rgbimage(filenamc) 
char filename!]; /* input filename */ 



{ 

register IMAGE * image; 

NPSIMAGE *img; /* ptr to an NPSIMAGE structure */ 

register ini x,y; /* temp indices for each line of 
data from the sgi image. */ 

long *ptr; /* temp pointer for each word of the 
NPSIMAGE long image. */ 

short rbuf[4096], /* temp arrays to hold scratch info for */ 
gbuf[4096], /* processing an sgi image */ 
bbuf[4096], 
abuf[4096]: 



/* open an sgi image */ 

if( (imauc=iopen(fiIenanie f v r» == NULL ) 

{ 

lprintf(sidcrr > ”read_sgi_rgbimage: can’t open input lile 9fs\n’ filename); 
rcturn((NPSI M AGE * iNULL); 

} 



if(imagc->zsi/e<3) 

( 

fprintf(sidcrr/Yead_sgi_ri»bimai:e: this is not an RGB image lileXn”); 
re lurn((N PS 1 M AG E * ) N l J LL ): 

) 



/* here we should allocate an NPSIMAGE */ 
if(image->zsize == 3) 

( 

/* just allocate an rgb image */ 

ime = t:ct_cmpiy_nib_npsimage(imagc->xsize,imagc->ysizc, filename); 

) 

else 

{ 

/* get an image with alpha */ 

img = ^cucmptv_rL T ba_npsimage(imagc->xsize,imagc*>ysizc, filename); 

} 



/* get a pointer to the NPSIMAGE longs */ 
ptr = img->imgdata.biisptr; 
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/* for each row of ihe image ... */ 
for(y=0; y < img->ysizc; y=y+l) 

{ 

/* read a row of reds */ 
geirow(image,rbuf,y,0); 

/* read a row of greens */ 
getrow(imagc,gbuf,y, 1 ); 

/* read a row of blues */ 
getrow(image,bbuf,y,2); 

/* if we have an image with alpha, get ii */ 
if(img->typc == RGBAWITHALPHA) 

{ 

geirow(image,abul,y,3); 

} 

/* we must now step across the row and set each long integer 
of the NPS1MAGE formal by combining the info from the sgi 
rows. 

*/ 

for(x=0; x < img->\si/.c; x=x+l) 



/* compute the RGBa long to plug in and plug it in */ 
if(img->tvpc == RGBAWITHALPHA) 

I 

^ ptr = rhu! [ x | -r (gbullx] << 8) + (bbul[x] « 16) + (abulfx] « 24); 



/* RGBA image with alpha forced to Oxff */ 

*ptr = rbul'l x 1 + (gbuf[x] « 8) + (bbuf[x] « 16) + (Oxff « 24); 



step the ptr to the next long *“/ 
ptr++; 

} /* end for 



r we have set all the bytes of the image. 

Now return the ptr to the NPS1MAGE structure. 
*/ 

rcturn(img): 

} 
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/* 7 

/* void write_sgi_rgbimagc (char filenamcU, NPS1MAGE *img) 

/* 

/* The following function wiles an NPSIMAGE to a file. 

/* - coutesy of M. Zyda, Naval Postgraduate School 

*/ 

void write_sgi_rgbimage(filcname, img) 

char filename!]; /* output filename */ 

PS1MAGE *img; /* ptr to an NPSIMAGE structure 7 

{ 

register IMAGE * image; /* a ptr to an sgi image structure 7 

register ini \,y; /* temp indices for each line of data from the sgi image. 7 

long *ptr; /* temp pointer for each word of the NPSIMAGE long image. 7 

long dimen: /* dimension of this image V 

short rbuf[4096], /* temp arrays to hold scratch info for 7 
gbuf[4096], /* processing an s m image 7 
bbuf[4096] . 
abuf[4096]: 



/* set the dimension and zsize of this imauc 7 
if(inig->type == RG B AW 1TH ALPHA) 

( 

dimen = 4; 

} 

else 

{ 

r RGBA image witiioul alpha 7 
dimen = 3; 

) 



/* open an sgi rgb image for writing 7 

image=iopen(filename, , 'w ,, ,RLE(l), dimen, img->xsize,img->> size, dim en); 

/* get a pointer to the NPSIMAGE longs 7 
ptr = img->imgdata.bilspir: 

/* for each row of the image ... 7 
for(v=0; > < ima->ysize: v=y+l) 

{ 



r we must now step across the row and decode each long integer 
of the NPSIMAGE format into the 16 bit shorts sgi requires */ 

for(x=0: ,\ < imsi->xsi/.e; x=x+l) 

{ 

/* get die colors from the longs */ 
rbuflx] = ’ptr & OxOOOOOOff; 
gbuf[x] = (*ptr &. OxOOOOfrOO) » 8; 
bbuf|x] = (*ptr & OxOOffOOOO) » 16; 
abuf[x] = (*ptr & OxffOOOOOO) » 24; 
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/* step the pir to the next long */ 
ptr++; 

} 

/* write a row of reds */ 
putrow(image,rbuf,y,Oj; 

/* write a row of greens */ 
putrow'(image,gbuf,y,l); 

/* write a row' of blues */ 
putrow(image,bbuf,y,2); 

/* write a row' of alphas, if any */ 
if(img->typc == RGB AW1TH ALPHA) 

{ 

putrow(image,abul,y,3); 

} 

} 

/* we must close the output sgi image file */ 
iclosc(imagc): 
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E. FILE: EDGESL'PPOKT.H 



/* FILENAME: edgesupport.h 
/* AUTHOR: Kevin Peterson 
/* DATE: 29 January 1992 
/* 

/^DESCRIPTION: Collection of edge finding functions. 

/* 

/* void fatal(chur message) 

/* int gradicnt_anglcs_close(double r, double s) 

/* int close jo_ncgativc_pi(double phi) 

/* int horizontal(EDGE *r) 

/* EDGE *crcate_edgc(long z,double phi) 

/* void add_pixel_to_edge (long z, double phi, EDGE *r) 

/* EDGE *combine_edgcs(EDGE *rl, EDGE *r2) 

/* IMGJJNE * create Jine (EDGE *r, double M20, double Ml 1, double M02, 

/* double Dmajor, double Dminor, double Rho) 

/* void line_te$i (EDGE *r) 
r void chcck_activc_cdges () 

/* void rgbalong_to_bwlong (long rgbalong, long *bwlong) 

/* void pixel_meinbership (long z, double phi) 

/* void set_pixcl_whitc (long ’‘'rgbalong) 

/* void set_pixcl_black (long *rgbalong) 

/* void writc_all_lines (long x, long y) 

/* IMGJJNE Mastlines (NPSIMAGE *img) 

/^c 5fc 3fc If Ifc If: If if If > If If >. If If If If If >. ;• If S». If If If If If If If If If If If * If If If 5ft If If If If If If If If If If If If If If If If If If If If If If If If If * If If If if; If If 



/* Gradient Magnitude Threshold - for fastlines(img) */ 
^define THRESHOLD 30000.0 



/* Gradient Angular Orientation */ 

^define MAX_DELTAJPH1 0.5 /* maximum difference (in radians) */ 

/* Constants for function: void line_tcst (EDGE *r) */ 

#define MIN_PIXELS_PER_LINE 60 /* minimum pixels allowed for a IMGJJNE */ 
#define MIN_DMAJOR 20.0 /* minimum major axis length allowed */ 

#dcfine MAX_RHO 0.1 /* maximum ratio (Rho=Dminor/Dmajor) */ 

#dcfinc PI 3.141 59265 



/* --- Global variables - */ 

long Xdim; /* width of input image (nr pixels) */ 
long Ydim; /* w idth of input image (nr pixels */ 

int Linecount = 0: /* counter for number of IMG JJNEs made */ 



/* Pointers to: Present row EDGE list, Past row EDGE list, IMGJJNE list*/ 

EDGE *Present_cdgeJistJicad = NULL, 

*Present_cdgcJist_tail = NULL, 

*Past_edgcJistJiead = NULL: 

IMGJJNE *LineJistJtead = NULL; 
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/* */ 

/* void fatal (char message) 

/* 

/* Prints error message and exits out of the program. 

/* */ 

fatal(message) 
char ^message; 

( 

fprintf(stderr, "Fatal ERROR: "); 

perror(message); 

exit(-l); /* exit by failure */ 

) 



/* */ 

/* int gradicnt_angles_close (double r, double s) 

/* 

/* Returns 1 if gradient angle orientations of EDGEs r and s 
/* are within MAX_DELTA_PH1. 

/* Returns 0 otherwise. 

/* */ 

int gradient_angles_closc(r,s) 
double r,s; 

i 

if((r < ,\1AX_DELTA_PH1 - PI) && (s > 0.0)) r = PI + PI + r; 
else il((s < \1AX_DELTA_PH1 - PI) && (r > 0.0)) s = PI + PI + s: 

rciurn(fabs(r - s) < MAX_DELTA_PH1); 



/* */ 

I' int close_io_neuati\e_i)i (double phi) 

/* 

/* Returns 1 if orientation phi is within N1AX_DELTA_PH1 to -PI. 
/* Returns 0 oiherw ise. 

/* */ 

i nt c lose_io_n ega ti ve_p i (ph i ) 
double phi: 

{ 

return(Pl + phi < MAX_DELTA_PHI); 

) 



/* */ 

/* int horizontal (EDGE ’c) 

/* 

/* Returns 1 if orientation of EDGE e (r->avg_phi) is within 
/* MAX_DELTA_PHl/4 to PI/2 or -Pl/2 (the normal orientations for 
/* a horizontal line). 

/* Returns 0 otherwise. 

/* V 

int horizontal(e) 

EDGE *c: 

( 

double maxphi = MAX_DELTA_PH1 /4.0; 

return((fabs(e->avg_phi) > (0.5*PI)-maxphi) &.& 

(fabs(e->avg_phi) < (0.5*PI)+maxphi)); 
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V 



/* 

/* EDGE *create_edgc (long z, double phi) 

/* 

/* Returns pointer to an newly instantiated EDGE with variable 
/* based upon the input pixel z and orientation phi of pixel z. 

/* */ 

EDGE *create_edge(z, phi) 
long z; /* zih pixel in image */ 
double phi; /* gradient orientation of pixel z */ 

( 

EDGE *r; 

long x = z%(Xdim-2), /* (x,y) coordinates of pixel z in 2D image */ 
y = z/(Xdim-2); 

/* allocate memory for EDGE r */ 

if((r = (EDGE *)malloc(sizeof(EDGE))) == NULL) fatal (“create_edge; mallocNn”); } 

/* else initialize fields of EDGE r */ 

r->active = 0; 

r->first_pixel = z: 

r->last_pixcl = z; 

r->xmin = x; 

r->xmax = x; 

r->avg_phi = phi: 

r->sum_phi = phi: 

r->m00 = 1 : 

r->ml0 = x: 

r->m0l = y; 

r->ml 1 = x*y: 

r->m20 = x*x; 

r->m02 = y*y: 

r->next = NULL: 

if(Prescnt_cdge_hst_head == NULL) 

Present_edgc_list_head = r: 
if(Present_cdge_list_inil != NULL) 

Prescnt_cdgcjist_tail->nc\t = r; 

Present_edge_list_tail = r: 

return(r); 
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/* 7 

/* void add_pixel_to_cdgc (long z, double phi, EDGE *r) 

/* 

/* Updates EDGE r with new pixel z and orientation phi of pixel z. 

/* 7 

add_pixel_to_edgc(z, phi, r) 
long z; /* zth pixel 7 

double phi; /* gradient orientation of pixel z 7 
EDGE *r; /* EDGE to add pixel z to 7 

{ 

long x = z%(Xdim-2), /* (x,y) coordinates of pixel z in 2D image 7 
y = z/(Xdim-2): 



if((r->avg_phi > 0.0) && closc_to_negative_pi(phi)) 

{ 

phi = PI + phi + PI; 

) 

else if((phi > 0.0) && closc_Lo_negative_pi(r->avg_phi)) 

{ 

r->avg_phi = PI + r->avg_phi + PI; 
r->sum_phi = r->avti_phi * r->m00; 

) 



/* update the nev\ xmax for this row and lasi_pixel added to this EDGE 7 
r->xmax = x; 
r->last_pixcl = /: 

/* update the least squares fit moments 7 

++r->m00; 

r->ml0 += x; 

r->m01 += y; 

r->ml 1 += x*y: 

r->m20 += x*x: 

r->m02 i-= y*v; 

/* recalculate the average gradient orientation for the EDGE 7 

r->sum_phi += phi; 

r->avg_phi = r->sum_phi / r->m()0; 

} 
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/* V 

/* EDGE *combine_edges (EDGE *rl,EDGE *r2) 

/* 

/* Returns pointer to EDGE rl after appending all information 
/* of EDGE r2 with rl . 

/* */ 

EDGE *combine_edges(rl,r2) 

EDGE*rl,*r2; 

{ 



/* Special case where rl->first_pixel is changed: */ 

if(!((r2->first_pixel > rl->first_pixel) 11 (horizontal(rl) && (rl->xmin < r2->xmin)))) 

{ 

rl->first_pixel = r2->first_pixcl; 



/* Modify lasi_pixcl */ 
if(r2->last_pixel > rl->last_pixel) 

( 

r 1 ->lasi_pixcl = r2->last_pixel; 
rl->xmax = r2->xma\: 

) 

else if(horizontal(r2) (r2->xmax > rl->xmax)) 

{ 

rl->lasi_pixel = 1 2-> lasi_pixel ; 



/* Special case: if gradient angles on opposite sides of +/- pi */ 
if((rl->avg_phi > 0.0) && close J.o_ncgativc_pi(r2->avg_phi)) 

r2->avg_phi = PI + r2->avg_phi + PI; 
r2->sum_phi = r2->a\ej)hi * r2->m00: 

) 

else if((r2->a\a_phi > 0.0) close_to_negative_pi(rl->avg_phi)j 

{ 

rl->avg_phi = PI + rl->avg_phi + PI; 
rl->sum_phi = rl->avir_phi * rl->m00; 

) 



r combine least squares lit moments */ 

rl->m00 -i-= r2->m00: 

rl->ml0 += r2->ml(): 

rl->m01 += r2->m()l: 

rl->ml 1 += r2->ml 1; 

rl->m20 += r2->m20; 

rl->m02 += r2->m02: 

/* combine and recalculate average gradient orientation lor the EDGE */ 
rl->sum_phi += r2->sum_phi; 
rl ->avg_phi = rl ->sum_phi / rl ->m(X); 

return(rl ); r return EDGE rl with combined information of rl & r2 */ 



91 



/* */ 

/* IMG_LINE *crcaie_linc (EDGE *r, double M20, double Mil, double M02, 
/* double Dmajor, double Dminor, double Rho) 

/* 

/* Returns pointer to newly instantiated IMG_LINE with variables set 
/* according to moments described by EDGE r, secondary moments 
/* M20, Mil, and M02, axis lengths Dmajor, Dminor, and ratio of 
/* axis lengths Rho. 

/* -*/ 

IMG_LINE *crcatc_linc(r,M20,Ml 1,M02, Dmajor, Dminor, Rho) 

EDGE *r; 

double M20,M1 1,M 02, Dmajor, Dminor, Rho; 



IMGJJNE *1; 

/* r->first_pixel mapped onto the IMGJJNE will be endpoint pi 
r->last_pixel mapped onto the IMG_LINE will be endpoint p2 */ 

long xl = r->firsi_pixel%(Xdim-2), 
yl = r->first_pixel/(Xdim-2), 
x2 = r->last_pixel%(Xdim-2), 
y2 = r->last_pi\cI/(Xdim-2); 



r Calculate the normal orientation of the IMG_LL\ T E by atan2() function. */ 

double Phi = uuui2(-2*M 1 1 ,M02-M20)/2.0, 

/* Della 1 and deka2 are the offsets used to calculate the endpoints 
lor the 1MG_L1NE segment based upon values xl,yl and x2,y2. */ 

delta 1 = (r->ml0/r->m00 - (doublc)xl)*fcos(Phi) + 

(r->m01/r->m()0 - (double)yl)*fsin(Phi), 
delta2 = (r->ml0/r->m()0 - (double)x2)*fcos(Phi) + 

(r->m()l/r->m00 - (double)y2)*f$in(Phi); 



/* Phi = atan2(-2*M 1 1 ,M02-M20)/2.0) always returns positive result to Phi. 

Therefore, ncgative_phi = (r->avg_phi < 0.0) is necessary. */ 

int ncgative_phi = (r->avg_phi < 0.0); 

r Allocate memory for IMGJJNE 1. */ 

if((l = (1MG_L1NE * }malloc(si/eof(IMG_LINE))) == NULL) fatalO‘create_line: mallocNn”); 



/* Calculate x,y coordinates for endpoints pi and p2. */ 

l->pl.x = (double)xl + dcltal*fcos(Phi); 
l->pl.y = (double >\ 1 + dcltal*fsin(Phi); 

L>p2.x = (doublc)x2 + dclta2*fcos(Phi); 

L>p2.y = (double)\2 + dc!ia2*fsin(Phi); 
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/* Copy least squares fit moments. */ 



l->mOO = r->m00; 
l->mlO = r->mlO; 
l->m01 = r->m01; 
l->mll = r->ml 1; 
l->m20 = r->m20; 
l->m02 = r->m02; 



/* Phi is positive, but -pi < r->avg_phi < pi. */ 

if(negative_phi) l->phi = -Phi; 
else l->phi = Phi: 



/* Update rest of 1MG_LINE values. */ 
l->next = NULL; 



l->dmajor = Dmajor; 
l->dminor = Dminor: 
l->rho = Rho; 

l->matchlist = NULL: 
l->pm = NULL: 



++Linecount; / > Increment global variable, Linecount. */ 
strcpy(l->name, v “); 
sprintf(l->namc, “9rd’\ Linecount); 
return(l); /* Return 1MG_L1NE 1. */ 

} 
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/* 

/* void line_test (EDGE *r) 

/* 

/* Determines if EDGE r meets three requirements to be a IMGJLINE: 

/* (1) The number of pixels in EDGE r (r->m00) be greater than MIN_PIXELS_PER_LINE. 
/* (2) The ratio (Rho) of the length of major and minor axes of the 
/* EDGE be less than MAX_RHO. 

/* (3) The length of the major axis (Dmajor) be greater than MIN_DMAJOR, the minimum 
/* IMG.LINE length allowed. 

/* If all three conditions are met, a new IMG_LINE type is created and 
/* appended to the Line Jist in order of significance (in this case Dmajor). 

/* */ 
line_test(r) 

EDGE *r; 

{ 

IMG_LINE *1, *in$crt_pt = Line_list_head; 

double M20,M 1 1 ,M02,Ma,Mb,Mmajor,Mminor,Dmajor,Dminor,Rho; 

/* First test -- A 1MG_LINE must have a required minimun number of pixels. */ 
if(r->mOO > M1N_P1XELS_PER_LINE) 

( 

/* Calculate secondary moments by least squares fit. */ 

M20 = r->m20 - ((r->mlO*r->mlO)/r->mOO); 

Ml 1 = r->ml 1 - ((r->ml0^r->m01)/r->m00); 

M02 = r->m()2 - ((r->m()l*r->in01)/r->m00); 

/* Calculate major and minor axis lengths, Dmajor and Dminor. */ 

Ma = (M20+M02)/2.0; 

Mb = sqrt( ((M02-M2Q)*(M02-M20)/4.0) + (Ml 1*M 11) ); 

Mmajor = Ma - Mb; 

M mi nor = Ma + Mb; 

Dmajor = 4.0*sqrt(Mminor/r->m00); 

Dminor = 4.0*sqrt(Mmajor/r->m00); 

/ * Calculate ratio Rho. *7 
Rho = Dminor/Dmajor: 

/* Second & Third tests - Ratio Rho must represent not a blob. 

- IMGJJNE must be at least a certain length. *j 
if ((Rho < MAX_RHO) && (Dmajor > MIN_DMAJOR)j 
{ 

/* The EDGE passed the three requirments to be a line. */ 

1 = creaic_line(r,M20,Ml 1,M02, Dmajor, Dminor, Rho); 

/* Add new 1MG_LINE to IMGJJNE list in order by IMG_LINE length, dmajor. */ 
if(Linc_list_hcad == NULL) Line_hsi_head = 1; 
else if(l->dmajor > Linc_list_head->dmajor) 

( 

l->next - Linc_list_hcad; 

Linc_list_hcad = 1; 

) 

else 

{ 

while((insert_pt->next != NULL) && (l->dmajor< inscrt_pt->next->dmajor)) 
insert_pt = inscrt_pt->next; 
l->ncxt = insea_pt->next; 
insert_pt->next = 1; 

} 

} /* end il second and third tests */ 

) /* end if first test */ 

i 
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y* * j 

/* void check_active_edges () 

/* 

/* This function, performed ai the start of scanning for each row, 

/* cycles through the EDGEs in the Present_edge_iist to see if: 

/* (1) Adajaccnt EDGEs on the same row have gradient orientations 
/* dial are close and therefore can be combined into one 
/* EDGE. 

/* (2) EDGEs from the Present_edge_list are adjacent to 
/* EDGEs from the past row’s Previous_edge_list and their 
/* gradient orientations are close so that they may be 
/* combined into one EDGE. 

/* The second method for determining when to combine_edges() 

/* provides the means for constructing EDGEs across rows. 

/* 

/* All EDGEs not combined w ith a EDGE from the past row are 
/* tested by linc_tcst() which examines the EDGE for satisfying 
/* the IMG_LINE requirements and appends the IMG JJNE to the Line_list. 

I* * j 

c he ck_ac 1 1 vc_ed ecs() 

{ 

EDGE *prcs = Prcsent_edgc_list_head, /* the EDGEs found by the scan of this row */ 

*pasi = Past_edge_list_head, /* the EDGEs found during the scan of the previous row */ 
*tcmp; /* a temporary pointer used for freeing memory */ 

int continue Joop; /* a boolean integer */ 



/* Loop through all EDGEs in the Prescnt_edge_list (all EDGEs found 
during the scan of the present row. */ 

while(prcs != NULL) 

{ 



/* Look forward to the next EDGEs on the present row to see if 
any should be combined with this EDGE. If so, combinc_edgc$0 
and free() the second EDGE. */ 

whilc((pres->nexi != NULL) &.&. (prcs->xmax+l == pres->next->xmin) && 
eradicni_aimlcs_closc(pres->avg_phi,prcs->ncxt->ave_phi)) 

{ 

pres = combine_edges(pres,prcs->next); 
temp = prcs->next; 
prcs->nexi = pres->next->ncxt; 
free(temp): 



/* Set continuejoop boolean to true and loop through the EDGEs of 
found during the scan of the previous row' of pixels. */ 
continuejoop = 1: 

while((past != NULL) (pres->xma\ > past->xmin-2) && (continuejoop)) 
( ' 



/* If the two EDGEs are adjacent and Lheir gradient orientations 
are close, then the information of the past EDGE must be in- 
cluded with the present EDGE and the EDGE should be marked 
that it is still "active". */ 
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if((pres->xmin < past->xmax+2) && gradient_antdes_close(pasi->avg_phi,pres->avg_phi)) 

( 

pres = combine_edges(pres,past); 
pasi->aciivc = 1 ; 



/* If the past EDGE was not appended to a present EDGE, (i.e. 
not “active”) then no more pixels may be added to it and it 
shall be tested if it satisfies the conditions for an IMG_LINE. */ 

if((past->next 1= NULL) && (pres->xmax > past->next->xmin-2)) 

{ 

il(!pasi->aciivc) line_test(past); 
past = past->next; 



/* If the past EDGE did not meet any of the above two conditions, 
then exit this loop and continue evaluation with the next 
EDGE of the present row. */ 

else continue Joop = 0; 

} 



pres = pres->next: 

} 



/* Continue line_testO and frce() all EDGEs in the past row. */ 
while(past != NULL) 

{ 

temp = past: 

if(!past->active) lme_test(.past); 
past = past->nexi: 
free(temp): 

) 



/* Finally: Past row <- Present row , Present row <- empty. *7 
Past_edge_list_head = Present_edge_list_hcad; 
Present_edgejistjiead = NULL; 

Prcscni_edge_lisi_iail = NULL: 
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j * * j 

I* void rgbalong_to_bwiong (long rgbalong, long *bwlong) 

/* 

/* Converts color to black/white for rgba formatted pixels. 

/* The weights assigned for each color are television standards. 

/* This function courtesy of M. Zyda. 

/* */ 

rgbalong_to_bwlong(rgbalong,bwlong) 
long rgbalong; /* input color rgbalong */ 
long *bw long; /* output b/w rgbalong */ 

{ 

unsigned char red, green, blue, alpha; 
unsigned bw; 

/* Use bit masks to get RGB and alpha values from input rgbalong. */ 

red = rgbalong & OxOOOOOOff; 

green = (rgbalong & 0x00001X00) » 8; 

blue = (rgbalong & OxOOffOOOO) » 16; 

alpha = (rgbalong & OxffOOOOOO) » 24; 

/* Calculate the black&whiie intesity using NTSC standard, 
intensity = 0.299(rcd) + 0.587(green) + 0.1 14(blue) */ 
bw = (0.299* red)+(0.587*grcen)+(0.1 1 4^ blue): 

/* Save the black&white intensity in bwlong. */ 

*bwlona = talpha«24)l(b\\«16)l(bw«8)lbw; 

) 



/*— */ 

/* void pixcljnembership (lone z, double phi) 

/* 

/* For a given row while scanning the black& white image, (scan of row> ....yyyyy....rrrrrrrrrz 
/* y = pixels included in pre\ious EDGE of same row, 

/* r = pixels included in EDGE r, 

/* z = pixel / beina tested for inclusion with EDGE r) 

/* 

/* tw o requirements must be satisfied for a pixel (z) with the pixels of a given EDGE: 

/* (1) pixels must be adjacent (i.e. the lasi_pixel of the EDGE + 1 must equal pixel z), 

/* (2) die gradient angle orientations of the lasi_pixel and z must be “close”. Otherwise, pixel z 
/* can not be included with EDGE rand must be considered the first_pixel of a new' EDGE for this row. 
/* */ 
pixel_membership(z, phi) 

long z: /* the zth pixel of the gradient image */ 
double phi: 

l 

EDGE *r: 

if((Prcsent_cdgc_lisi_tail != NULL) && 

(Prcsent_edgc_list_tail->lasi_pixel+l == z) && 
gradicni_angles_close(Preseni_edge_lisi_tail->avg_phi,phi)j 
{ 

add_pixel_to_cdiie(z,phi,Present_cdge_list_tail); 

} 

else r create new EDGE and add to end ol Present list */ 

{ 

r = crcaie_cdgc(z,phi): 

i 

) 
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/* */ 

/* void set_pixel_whitc (long *rgbalong) 

/* 

/* Sets the specified long integer pointed to by rgbalong to be 
/* white by setting all RGB bits to ff (255 dec -> max intensity). 
/* */ 
set_pixe!_whiie(rgbalong) 
long *rgbalong; 

{ 

* rgbalong = Oxl 1111 ft 1 ; 

} 



/* */ 

/* void set_pixel_black (long *rgbalong) 

/* 

/* Sets the specified long integer pointed to by rgbalong to be 
/* black by setting all RGB bits to 00 (0 dec -> min intensity). 

* j 

set_pixel_black(rgbalong ) 
long * rgbalong; 

{ 

*rebalone = 0x11000000; 

} 



/* */ 

/* void write_all_lines (lone \, long y) 

/* 

/* W rite to output file "name. text’'. 

/* — *7 

writc_all_lines( x,yj 

lone x,y: /* the x,\ dimensions of the image */ 

[ 

1MG_L1NE *1 = Line listjiead; 

FILE *lines_file: 



lincs_l ile = fopen(“lines.text , \ ,, w”j; 

fprinif(linesj‘ik\" c ,'f d (vd\n\n’\x,y); 

while(l!=NULLj 

{ 

fprinif(lines_file,”9( .2f %.2f\n%.2f %.2(\n%.4f\n”, 
l->pl.x,l->pl.y,l->p2..\,l->p2.y,l->phi): 

1 = l->nc\t: 

) 

fclosc(lines_t'ilc): 

priml (” lines found in nmme w riucn to: l lines.text’\n”); 

} 
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/* 

/* IMG.LINE * las il i nes (NPS IMAGE) 

/* 

/* Returns a pointer to the head of a linked list of vertical IMGJJNEs found in an NPSIMAGE. 
/* */ 

IMG_LINE * fasti ines(img) 

NPSIMAGE *img; /* input file_name.rgb color image */ 

{ 

long *ptr; /* pointer to bitsptr of NPSIMAGE */ 
long grmask = OxOOOOffOO; 
double dx, dy, Th = THRESHOLD*THRESHOLD; 
register int 1 = 0,/* counter for pixels in input image */ 

/. = 0; /* counter for pixels in gradient image */ 

EDGE *reg; 



Xdim = img->xsize; 

Ydim = img->ysize; 
ptr = img->imgdata.bitsptr; 

Linecount = 0; 

/* The scan of an RGB image is from the bottom row -> up, traversing the rows left to right. */ 
for(i = Xdim + 1; i < (Xdim*(img->ysize-l))-l; ++i) 

{ 



/* If pixel i is in the leftmost column, do chcck_activc_cdgcs(). */ 
if(i%Xdim == 0) check_activc_edges(); 



/* Ensure pixel i is not in leftmost or rightmost column */ 
else if(i Xdim != Xtlim-1 ) 

( 

/* Calculate dx,dy via Sobcl operator for pixel i. */ 

dx = (-(pir[i+Xdim-J J&grmask) + (ptr[i+Xdim+l]&grmask) 
-(2 * (ptrfi-1 ]&grmask))+ (2 * (ptrfi+l]&grmask)) 

-t ptr[i-Xdini- 1 RLgrmask) + (ptr[f-Xdim+l]&grmask)); 

d) = ( (ptr[i+Xdim-l]&grmask) + (2*(ptr[i+Xdim]&grmask)) 
+ (ptr[i+Xdim+l ]&grmask) - (ptr[i-Xdim-l]&grmask) 

- (2*(pu[i-Xdim]&grmask)) - (ptr[i-Xdim+l]&grmask)); 

if((dx*dx)-t-(dyNi\ ) > Th) pixel_membership(z,atan2(dy,dx)); 

} 

} /* endfor i */ 

/* Check remaining EDGEs for lines: */ 
reg = Past_cdge_list_head: 
while(rcg != NULL) 

{ 

hnc_test(rcg): 
reg = reg->next: 

} 



/* Write the lines list to file “lines. text”. */ 
write_all_lincs(img->xsize,img->ysize); 

printf(“ Number lines found in %s = %d\n”,img->name, Linecount); 
return(Line_list_head): 

i 
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FILE: DISPLA YSUPPORT.H 



/* FILENAME: displaysupport.h 
/* AUTHOR: Kevin Peierson 
/* DATE: 31 January 1992 
/*DESCRIPTION: Collection of display functions. 

/* 

/* void display_bw_and_gradient_images (NPSIMAGE *nngl, NPSIMAGE *img2, 

/* ' IMG_LINE *1) 

/* void draw_red_lincs (IMG_LINE *1) 

/* void display Joop (NPSIMAGE *imgl, long winidl, 

/* NPSIMAGE *img2, long winid2) 

//* void displayjinejmage (NPSIMAGE *img, IMG_LINE *1) 

/* void display_linc_loop (NPSIMAGE *img, IMG JLINE *1, long winid) 

/* 

/* These functions make calls to following SiliconGraphics routines: 

/* prefsize(), winopcn(), w inset(), winclose(), RGBmode(), 

/* singlebuffer(), gconfigO, qdevice(), c3f(), move2(), draw2(), 

/* swapbuffersO. lrectwritcQ, reshapeviewport(), and clear(). 

/* */ 

/* void display_bw_and_tiradicnt_images (NPSIMAGE *imgl, NPSIMAGE *img2, 

/* ~ 1MG_LINE *1) 

/* Displays NPSIMAGE imgl and img2 on SiliconGraphics’ Iris workstation. 

/* Each window will be displayed w ithin red outline when operator depresses left mouse button. 

r v 

display J)w_and_gradicnt_imagcs(imgl,img2,l) 

NPSIMAGE *imgl,/* input b/w image */ 

*irm*2; /* gradient image */ 

IMGJJNE *1: 

{ 

long winidl, \vinid2: r silicon graphics window id’s */ 

prcfsizc(imgl->\sizc,imgl->ysizc); /* preferred size for window */ 
winidl = w'inopen(imgl->namc); /* open the window */ 

RGBmode(): /* set RGBmode, singlebuffer, and */ 
singlcbuffer(J; /* configure the w indow' */ 
gconfigO; 

prefsizc(img2->xsize,img2->ysize); /* preferred size for w indow */ 
w inid2 = w inopen(img2->name); /* open the window' */ 

RGBmodcO; /* set RGBmode, singlebuffer, and */ 
singlebuffer(): /* configure the w indow */ 
gconfigO; 

/* initialize_controls '•/ 

qdeviee(REDRAW); 

qdevice(LEFTMOUSE); 

qdeviee(MIDDLEMOUSE): 

qdcvicc(RIGHTMOUSE); 

qdeviec(ESCKEV): 

display_loop(img 1 ,w inid 1 ,img2,w'imd2,l); 

free(imgl->imgdata.bitspu-;; /* delete the bitmap for the image */ 

freezing 1); /* delete the NPSIMAGE structure */ 

w'inclosc(w'inidl); /* close the w'indow' */ 

frce(img2->imgdata.bitsptr); 

frce(img2); 

w'inclose(w mid2): 
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/* 

/* void draw_redJines(IMG_LINE *1) 

/* 

/* Draws red lines over an image. 

/* : V 

draw_rcd_lines(l) 

IMG_LINE *1; 

{ 

sialic float red[3] = { 1 . 0 , 0 . 0 , 0.0) ; /* rgb red */ 

c3f(red); 

whilc(l!=NULL) 

( 

move2(l->p 1 .x,l->p 1 ,y); 
draw2(l->p2.x,l->p2.y): 

1 = l->nexi; 

} 

swapbulfersO: 

) 



/* */ 

/* void draw_whitc_lines(]MG_LINE *1) 

/* 

/* Draws while lines over an image. 

/* : */ 

draw_whiic_Ii»es(l) 

IMG_LINE ’1: 

( 

sialic float white|3) = { 1.0, 1.0, 1.0);/* rgb while */ 

c3f(whiic): 
while(l!=NULL > 

{ 

movc2(l->pl.x,l->pl ,\ ): 
draw2(l->p2.x,l->p2.\ ): 

1 = l->ne.xi: 

) 

swapbullcrs(): 

) 



/* *7 

/* void draw_black_lines(.lMG_LINE *1) 

/* 

/* Draw s black lines oxer an image. 

/* */ 

draw_black_linesd) 

IMG.L1NE *1; 

{ 

sialic float black[3] = (0.0, 0.0, 0.0); /* rgb black */ 

c3f(black); 

whilc(l!=NULL) 

( 

movc2(l->pl.x,l->pl.y); 

draw2(l->p2.x,l->p2.y); 

I = l->nexi; 

) 

swapbulfcrs(): 
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/* */ 

/* void displayjoop (NPSIMAGE *imgl, long winidl, 

/* NPSIMAGE *img2, long \vinid2) 

/* 

/* Continuously displays the white & white (imgl) and gradient (img2) 
/* images within their respective windows (winidl and winid2). 

/* - Depress left mouse button when cursor arrow is in title 
/* bar to sec lines initially over the image. 

/* - Depress middle mouse button when cursor is in image to 
/* see lines only. 

/* - Depress middle mouse button when cursor is in title bar 
/* ‘‘drag” window and redisplay image with lines. 

/* - Depress right or left mouse buttons when cursor is i 
/* either image to kill windows. 

/* ; : */ 

display_loop(imgl, winidl ,img2,winid2,l) 

NPSIMAGE *imgl, /* input b/w image */ 

*img2; /* gradient image */ 
long winidl , winid2; /* window id’s for images */ 

IMG.LINE *1: 



static float white[3J = ( 1.0,1. 0,1.0); /* rgb white */ 
short value; /* value returned from the event queue */ 

/* display the images once */ 
winset(winidl ); 

lrectwritc(0,0,imgl->.\si/e - l,imgl->ysize - l,imgl->imgdata.bitsptr); 
winsct(winid2); 

lrcctwrite(0,0,img2->xsi/e - l,img2->ysizc - l,img2->imgclaia.bitsptr); 

/* loop until a mouse button is pressed */ 
while(TRUE) 

( 

swiiclHqread(ck\alue) » 

[ 

case REDRAW: 

winset((long)valuej: 

reshapeviewport(): 

if(value == winidl) Irectwrite(0,0,iingl->xsize - l,imgl->ysize - 1, imgl->imgdata.bitsptr) 
i ('(value == winid2) lrectw ritc(0,0,img2->xsize - l,img2->ysi/e - 1, img2->imgdata.bitsptr) 
draw __rcd_l i nest lj: 
break; 

case LEFTMOUSE: 
case M1DDLEMOUSE: 
i Rvalue == 0) ( 
c 31 (white): 
clear(); 

draw_red_lmes(l); 

i 

break; 

case R1GHTMOUSE: 
i Rvalue == 0) return: 
break; 

case ESCKE\': 
exit(0); 
default: 
break; 

) /* end switch */ 

} /* end while *! 
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/* */ 

/* void display_line_image (NPSIMAGE *img, 1MG_L1N T E *1) 

/* * j 

displayjine__image(img,l) 

NPSIMAGE *img; 

IMG_LINE *1: 

{ 

long winid; 

prefsize(img->xsize,img->ysize); /* preferred size for window */ 
winid = winopen(img->name); /* open the window */ 
RGBmode(); /* set RGBmode, singlebuffer, and */ 
singlebufferQ; /* configure the window */ 
gconfig(); 

/* initialize_controls */ 

qdevice(REDRAW); 

qdevice(LEFTMOUSE); 

qdevice(MlDDLEMOUSE); 

qdevice(RlGHTMOUSE): 

qdevicc(ESCKEV): 

display _IineJoop(img,l, winid j: 

free(img->imgdata.bitsptr); /* delete the bitmap for the image */ 
free(img); /* delete the NPSIMAGE structure */ 
winclose(\\ mid); /* close the window */ 

} 



/* */ 

/* void displa>_line_loop (NPSIMAGE *img, IMGJJNE * 1, long winid) 

/* : — -*/ 

dispIay_line_Ioop(imeJ,w inid) 

NPSIMAGE *imm 
IMG_LINE *1; 
long winid; 



{ 

static float white[3] = ( 1.0, 1.0, 1.0); /* rgb white */ 
short value; /* value returned from the event queue */ 

/* display the images once *7 
winset(winid); 

lrectwritc(0,0,img->xsize - 1 ,img->ysize - 1 ,img->imgdata.bitsptr); 

/* loop until a mouse button is pressed */ 
while (TRUE) 



switch(qread(& val ue ) ) 

{ 



case REDRAW: 

winset((long)value); 

reshapeviewporiO; 

if(value == winid) lrectwriie(0,0,img->xsize - l,img->ysize - 1, img->imgdata.bitsptr); 
break; 
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case LEFTMOUSE: 

/* draw white lines over input image */ 
if(value == 0) draw_whitejines(l); 
break; 

case MIDDLEMOUSE: 
if(value == 0) 

{ 

c3i(white); 

clcar(); 

draw_black_hnes(l); 

} 

break; 

case RIGHTMOUSE: 
if(value == 0) return; 
break; 

case ESCKEY: 
cxit(0); 

default: 

break; 

} /* end switch */ 

J /* end while */ 
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G. 



FILE: VKKSLITORT.il 






/* FILENAME: vertsupport.h 
/* AUTHOR: Kevin Peterson 
/* DATE: 29 January 1992 
/* 



/*DESCRIPTION: Supplementary vertical edge finding functions. 

/* int vertical(EDGE) 

/* void vertical_Iine_tesi(EDGE) 

/* void check_active_vertical_edges() 

/* 1MG.L1NE *verilines(NPS IMAGE) 

/*********************************************************************** 



#define VERT_CONSTRAINT 0.0349 /* radians (= 2.0 degrees) */ 



/* V 

/* ini venical(EDGE) 

/* 

/* Returns 1 if orientation of EDGE r (r->avg_phi) is within 
/* MAX_DELTA_PHI/4 of 0, PI, or -PI (the normal orientations for 
/* a vertical line). 

/* Returns 0 otherwise. 

/* */ 

int vertical(r) 

EDGE *r: 

( 

return ((fabs(r->ave_phi) < V ERT_CON STR A INT) II 
(fubs(r->avg_phif> PI - VERT_CONSTR AIN'T)): 
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I* */ 

/* void vcriica]_line_lcsi(EDGE) 

/* 

/* Same as linc_tcsi(r), bui also ensures that EDGE r is vertical. 

/* */ 
vcrtical_line_tcst(r) 

EDGE *r; 

( 

IMG_L1NE "“l, *inscrt_pt = Line_list_head; 

double M20,M 1 1 ,M02,Ma, Mb, Mmajor,Mminor, Dmajor, Dminor, Rho; 

/* First test -- A IMG_LINE must have a required minimun number of pixels. */ 
if((r->m00 > M1N_P1XELS_PER_LINE) &X (vertical(r))) 

{ 

/* Calculate secondary moments by least squares tit. */ 

M20= r->m20 - ((r->mlO*r->mlO)/r->mOO); 

Ml 1 = r->ml 1 - ((r->ml0*r->m01)/r->m00); 

M02 = r->m02 - ((r->m01*r->m01)/r->m00); 

/* Calculate major and minor axis lengths, Dmajor and Dminor. */ 

Ma = (M20+M02)/2.0; 

Mb = sqrt( ((M02-M20r(M02-M20)/4.0) + (Ml 1*M1 1 ) ); 

M major = Ma - Mb; 

Mini nor = Ma + Mb: 

Dmajor = 4.()*sqrl(Mminor/r->m00); 

Dminor = 4.0*sqrt(Mmajor/r->mOO): 

/* Calculate ratio Rho. T 
Rho = Dminor/Dmajor; 

/* Second X Third tests - Ratio Rho must represent a line, not a blob. 

-- 1MGJJNE must be at least a certain length. */ 
il'((Rho < MAX_RHO) XX (Dmajor > M1N_DMAJ0R)) 

{ 

/* The EDGE passed the lour requirments to be a vertical line. */ 

1 = create Jme(r,M20, Ml 1 ,M02, Dmajor, Dminor, Rho): 

r Add new 1MG_L1NE to IMG JJNE list in order by 
IMG_LINE length, dmajor. */ 
if(LineJist_hcad == NULL) Line_list_head = 1; 
else if(l->dmajor > LineJist_hcad->dmajor) 

{ 

l->next = Linc_list_hcad; 

Line_list_head = 1; 



else 

{ 

while((insert_pt->nexi != NULL) XX 
(l->dmajor < inscri_pi->nexi->dmajor)) 
( 

insert_pt= insert_pt->next; 

) 

l->next = insert_pt->ne.\t; 
insert ju->ne.\t = 1: 

} 



} r end ll second and third tests */ 
) /* end if first test */ 

} 
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/* */ 

/* void check_aclive_vcrtical_edges() 

/* 

/* Same as check_aciivc_edges(), bui for vertical EDGEs only. 

/* */ 

check_aciive_veriical_edges() 

( 

EDGE *pres = Preseni_edgcjisi_head, 

/* ihc EDGEs found by the scan of this row */ 

*pasi = Past_edgejist_head, 

/* the EDGEs found during ihc scan of ihc previous row */ 

*iemp; /* a temporary pointer used for freeing memory */ 

ini coniinuejoop; /* a boolean integer */ 

/* Loop through all EDGEs in the Present_edge_list (all EDGEs 
found during the scan of the present row. */ 

while(pres != NULL) 

( 

/* Look lorward to the next EDGEs on the present row to sec il 

any should be combined with this EDGE. If so, coinbinc_edges() 
and freeQ the second EDGE. */ 



while((prcs->ne\i != NULL) && (pres->xmax+l == prcs->next->xmin) && 
eradieni_aneles_close(prcs->avg_phi,pres->ncxl->avg_phi)) 

r 

pres = combinc_edges(pres,pres->ncxi); 
temp = pres->ne.\t; 
pres->nexi = prcs->nc\t->nexi; 
free (temp): 

} 



/* Set coniinuejoop boolean to true and loop through the EDGEs ol 
lound during the scan of the previous row of pixels. 

coniinuejoop = 1: 

whilc((pasi != NULL) && (pres->xmax > pasi->xmin-2) && (continuejoop)) 



/* If the two EDGEs arc adjacent and their gradient orientations 
are close, then the information of the past EDGE must be in- 
cluded with the present EDGE and the EDGE should be marked 
that it is still “active". */ 

if((pres->\min < past->xmax+2) && 
irradicnt_angles_close(pasl->avg_phi,prcs->avg_phi)) 

( 

pres = combinc_edgcs(pres f pasi); 
past->active = 1; 



/* If the past EDGE was not appended to a present EDGE, (i.e. 
not “active”) then no more pixels may be added to it and it 
shall be tested if it satisfies the conditions for an IMG_LINE. */ 

if((past->next != NULL) && (pres->xmax > pasi->nexi->xmin-2)) 

{ 

il(!past->active) veriical_Iine_test(past); 
past = past->next; 
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r II the past EDGE did noi meet any of the above iwo conditions, 
then exit this loop and continue evaluation with the next 
EDGE of the present row. */ 

else cominuejoop = 0; 



pres = pres->next; 



/* Continue veriical_line_iest() and free() all EDGEs in the past row. */ 
while(past != NULL) 

{ 

temp = past; 

if(!past->active) vertical Jine_test(past); 

past = past->next: 

free(temp); 

} 



/* Finally: Past row <- Present row, Present row <- empty. */ 
Past_edge_lisl_hcad = Prcsent_edge_list_head; 
Present_edgcjist_head = NULL; 

Present_cdee_list_tail = NULL: 

} 
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/* */ 
/* IMGJJNE *vertlines(NPS IMAGE) 

/* 

/* Returns a pointer to the head of a linked list of vertical 
/* IMGJJNEs found in an NPSIMAGE. 

/* */ 
IMG_L1NE *vertlincs(img) 

NPSIMAGE *img; /* input file_name.rgb color image */ 

( 

long *ptr; /* pointer to bitsptr of NPSIMAGE */ 
long grmask = OxOOOOffOO; 
double dx, dy, Th = THRESHOLD*THRESHOLD; 
register int i = 0, /* counter for pixels in input image */ 
z = 0;/* counter lor pixels in gradient image */ 

EDGE *reg; 



Xdim = img->\size; 

Ydim = img->ysize; 

ptr = img->imgdata. bitsptr: 

Linecount = 0: 



/* The scan of an RGB image is from the bottom row -> up, 
traversing the rows left to right. */ 



lor(i = Xdim + 1: i < (Xdim*(img->ysize-l))-l; ++i) 

( 

r If pixel i is in the lelunost column, do check_activc_edges(). */ 
if(i%Xdim == 0) check_active_vertical_edges(); 

/* Ensure pixel i is not in leftmost or rightmost column */ 
else if(i%Xdim != Xdim-1) 

t 

/* Calculate d\,d> via Sobel operator for pixel i. */ 

d.\ = (-(pir[i+Xdim-l J&grmask) + (ptr[i+Xdim+l]&grmask) 
-(2 * (ptr[i-l |<5cgmiask))+ (2 * (ptr[i+l]&grmask)) 
-(ptr[i-Xdim-l ]& grmask) + (ptr[i-Xdim+l]&grmask)); 

(]) = ( (ptr[i+Xdnn-l]&grmask) + (2*(ptr[i+Xdim]&grmask)j 
+ (ptr[i+Xdim+l J&grmask) - (pLr[i-Xdim-l]&grmask) 

- (2*(ptr|i-Xdiml&grmask)) - (ptr[i-Xdim+l]&grmask)); 



if((d\*d\)+td)*d>) > Th) pixel_membcrship(z,atan2(dy > dx)); 
++z: 

} 

} /* endlor i * j 

/* Check remaining EDGEs for lines: */ 
reg = Pasi_edge_list_head; 
while(ree != NULL) 

{ 

vertical_line_test(reg ): 
reg = rcti->ncxi; 

} 



/* Write the lines list to file “lines. text”. */ 
wriie_alHincs(img->xsize,img->ysize); 

printf(“ Number lines found in %s = %d\n’\img->name, Linecount); 
return(Linc JistJiead ): 

} 
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APPENDIX C - LINE MATCHING AND POSE DETERMINATION ROUTINES 



A. IMPLEMENTATION: VERTMATCH.C 



/*********************:*. J***********************************************^ 

P FILENAME: vcrunatch.c 
/* AUTHOR: Kevin Peterson 
/* DATE: 09 March 1992 



#include <gl.h> /* SiliconGraphics (r) graphic library */ 

#include <gl/image.h> /* SGI image structure library */ 

^include <device.h> /* Machine-dependent device library for keys and mouse-buttons */ 
#include <stdio.h> /* C standard i/o library */ 
include <math.h> /* C math library for atan2() */ 



#includc “2d+.h” /* Header files for 3D model environment by J. Stein */ 
#includc “5th. h” 

^include “graphics. h” 

^include “visibility. h" 

#include “imagc_types.h” /* Type definitions for NPSIMAGE, etc. */ 
#include “match_types.h" p Type definitions for EDGE, IMG_L1NE.. */ 

#include “npsimagcsupport.h” p Some NPSIMAGE functions */ 

#include “edgesupport.h” p EDGE and IMG_LINE building functions */ 
^include “vcrtsupport.ir /■' Vertical EDGE and IMG JL1NE supplement */ 
#include “matchsupport.lf’ p LINE and IMGJJNE matching routines */ 
#includc “matchdispla) support. h*' p Graphics display functions */ 



main (arge, argvj 

int arge: 
char *aru\ | ): 

{ 

WORLD *FifthFloor; 

NPSIMAGE *img; 

IMG_LINE MmagcLinesHead = NULL, *imageline; 

LINE_HEAD * Model LinesHeadl = NULL, *ModelLinesHead2 = NULL; 

POSE *inj)ose, *corr_pose; 

float xO, y(), thetaO, u_posit, center_u_axis; 

if (arge != 2) fatal(“usage: \ertmatch filcnamcNn”); 

P Pose values for daui images: - - */ 

xO = 60.0; >0 = 366.0; thetaO = 250.0; P 060366250.pic */ 

P xO = 60.0; >0 = 366.0; thetaO = 253.0; p 060366253.pic */ 

P xO = 60.0; > 0 = 366.0; thetaO = 25 1 .0; P 06036625 1 .pic */ 

P xO = 59.0; >0 = 366.0: thetaO = 250.0; P 059366249.pic */ 

P x0 = 94.0; y() = 381.0; theiaO = 245.0; p 09438 1 245,pic */ 

P xO = 68.0; >0 = 372.0; thetaO = 249.0; p 068372249.pic */ 

P xO = 48.0; yO = 277.0; thetaO = 000.0; p Q48277000.pic */ 

P xO = 48.0; yO = 277.0; thetaO = 002.0; p 048277002.pic */ 

P xO = 48.0; yO = 277.0; thetaO = 356.0; p 048277356.pic */ 

/* xO = 43.0; yO = 277.0; thetaO = 356.0; P 043277356.pic */ 

P xO = 43.0; >0 = 267.0; thetaO = 000.0; p 043267000.pic */ 

P xO = 48.0; yO = 267.0; thetaO = 000.0; p 048267000.pic */ 

P xO = 48.0; vO = 272.0: thetaO = 000.0; p 048272000.pic */ 
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/* Read input rgb image and extract vertical edges. */ 
img = read_sgi_rgbimagc(argv[l]); 

if(img ==(NPS1MAGE *) NULL) fatal(“File %s is a NULL image.\n”,img->name); 
printf(“verimaich> %s xsize= %d ysize= %d pixels= %d\n”, 

img->namc,img->xsize,iing->ysize, (img->xsize*img->ysizc)); 

ImageLinesHead = vertlincs(img); 
imageline = ImageLinesHead; 
centcr_u_axis = (float)(img->xsize/2); 
while(imageline != NULL) 

( 

u_posit = (imageline->pl.x + imageline->p2.x)/2.0; 
imagcline->anglc_to_image_centcr = atan2((center_u_axis - u_posit), 
FOCAL_LENGTH_IN_PIXELS); 
imanelinc = imagcline->next; 

j 



/* Declare die estimated pose of the camera. */ 
if((in_pose = (POSE *)malloc(sizcof(POSE))) == NULL) 
fatal(‘ k creaiing POSE in_posc: mallocVT); 
in_pose->x = \0; 
in_posc->y = yU; 

in_posc->theta = normalize(thetaO * (PI/180.0)); 

printf(‘\nDR (input) pose: x = %.2f, y = %.2f, theta = %.2f(%.2f rads)\n\n’\ 
m_pose- > x,in_pose->y,thetaO,in_pose->theta); 



/* Initialize world database i’or fifth floor of Spanagle Hall, determine 
2D view of environment, and label the vertical model features. */ 

FifthFloor = make_workl(); 

ModelLinesHead 1 = eei_view(in_pose->.\, in_pose->y, CAMERA_HE1GHT, thetaO, 

FifthFloor, FOCAL JLENGTH); 

printf(“Nr Vertical Model Lines = %dVi”, ModelLinesHead 1->VERT_LINES); 
labcLmodel_lines(ModelLmesHead 1 ->VL1NE_LIST, in_po.se ); 



/* Call to update_poseQ */ 

corr_pose = update_pose(lmagcLincsHead,ModclLincsHeadl->VLlNE_LlST,in_pose); 

thetaO = corr_pose->theui * 1 80.0 / PI; 
if(iheiaO < 0.0) thetaO += 360.0; 

printlVViCorrected pose: \= %.2f, y= %2 f, thcta= %.2f(%.4f rads)\n”, 
corr_pose->x,corr_pose->y, thetaO ,corr_posc->theta); 

/* Get 2D view of world from corrected pose. */ 

ModelLincsHead2 = eet_\ iew(corr_pose->x, corr_pose->y, CAMERA_HEIGHT, thetaO, 

FifthFloor, FOCAL_LENGTH); 

/* Display image, 2D view from input pose, and 2D view from corrected pose. */ 
display jnatch_image(img. ImageLinesHead, ModelLincsHcadl, ModclLinesHead2); 



frecJincs(ModelLinesHeadl ); 
free_lines(ModelLinesHead2); 

printf(“ verimatch %s...done.\n”,argv[l]); 
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B. FILE: MATCHSUPPORT.H 



/* FILENAME: maichsuppori.h 
/* AUTHOR: Kevin Peterson 
/* DATE: 09 March 1992 

#dcfinc FOCAL_LENGTH 1.40/* cm */ 

#dcfine FOCAL_LENGTH_IN_PIXELS 1205.4524 /* number of pixels */ 
#dcfinc VERT_PICTURE_BORDER 8.0/* number of pixels */ 

#dcf'ine FIELD_HALF_ANGLE 0.2556/* radians = 14.645 degrees */ 

#dcfinc CAMERA_HEIGHT 40.0/* inches */ 

^define MlN_CHORD_LENGTH 10.0/* min disl between 2 vertical 

model features (in inches) 
used in dcicrmine_posiiion()*/ 

#dcfine DELTA_LEN'GTH_RAT10 0.05 /* Endpoint dclia/IMG_LINE length 

used in dctermine_vcri_maich()*/ 

#dcfinc IMG_LINE_MIN_ANGLE 0.0070/* radians = 0.20 degrees */ 

# define VERIFY. EQ_ A N G LE_EPS 1 LON 0.0035 /* radians = 0.20 degrees */ 

/* */ 

/* double normali/.e(,doble alpha) 

/* 

/* Returns an angle between pi and -pi. 

/* */ 

double normal i/.c( alpha) 
double alpha: 

{ 

whilcfalpha > PI) alpha -= (2.0* PI); 
whiletalpha < -PI) alpha += (2.0* PI); 
return alpha: 

} 



/* */ 

/* 

Returns 1 it' ancle a 1 is "loll of' angle a2. 

/* */ 

*/ 

ini lcfl_of(a 1 , a2) 

double a 1 , a2: 

l 

return (normali/e(al - a2) > 0.0); 

) 



/* */ 

/* 

Returns angle for use with environment model coordinates. 
Environment model uses ancles measured from y-axis. 

/* - */ 

*/ 

double map_angle(alpha) 
double alpha; 

{ 

return normali/.e(alpha - (Pl/2.0)); /* alpha.primc */ 

1 



112 



/* */ 

/* void label_modclJincs(LINE *m, POSE *p) 

/* 

/* Routine to determine lengths, orientations, angles from image center, and 
/* labels for all model LINEs visible in image. 

/* */ 

label_model Jincs(m, p) 

LINE *m; 

POSE *p; /* estimated camera POSE */ 

( 

unsigned ch:ir first_char = ‘ \ sccond_char = 1 A’; 



/* Note: Vertical edge extraction produces IMG_LINEs with pl.y always 
less than p2.y. Similarly for vertical model LINEs, YI is always 
less than Y2. 

*/ 

print! ("Nn Model Lines: \n”j; 

while(m != NULL) 

i 



m->length = m->Y2 - m->Yl; 

m->est_pose_oriem = map_anglc(atan2(m->MODEL_Y - p->y, 
m->MODEL_X - p->x)); 

m->est_angle_to_imagc_ccntcr = m->cst_posc_oricnt - p->thcta; 
il\n»length > 15.0) /* temporary - don’t do very small lines */ 



suvpy(m->namc/' “): 

sprinifon^name, ‘‘%c%c , \ first_char, second_char); 

ifi.second_char == ‘Z’) 

if(first_char == ‘ ‘) first_char = V A’; 
else first_char = first_char + 1; 
second_char = l A’; 

J 

else second_char = second_char + 1; 

printlV* c /cs: (%.lf,%.lf) length= %.2f, psi= %.2f, psi-thctaO= %.4f\n”, 
m->name, m->MODEL_X, m->\10DEL_Y, m->length, 
m->cst_posc_oricnt, m->est_anglc_ioJmagc_ccmcr); 




sprintf(m->namc, 44 “); 



m = m->NEXT: 



i 

j 
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* j 

/* void determine_vcrt_match(LINE *m, 1MG_LINE *i) 

/* 

/* Function to determine confidence value for a possible match 
/* based upon translation, rotation, scaling, and overlapping 
/* of endpoints between a LINE from the model list and an 
/* IMG LINE from the image list. 

/* . 1 . */ 

detcrmine_vert_match(m,i) 

LINE *m; /* ptr to LINE from model list */ 
1MG_L1NE *i; /* ptr to 1MG_LINE from image list */ 

{ 

int endpointjnclusion; 
double delta; 

MATCHTYPE * match, ’“place; 



/* Endpointjnclusion of image endpoints: for vertical image lines, i->pl is 
always lower than i->p2. */ 

delta = (m-> Y 1 - i->p 1 .y) + (i->p2.y - m->Y2); 

if((dclia <= 0.0) II ((delta > 0.0) && (dclta/i->dmajor < DELTA_LENGTH_RAT10))) 

{ 

r then create MATCHTYPE and place in the IMG_LlNE’s matchlist */ 

if(( match = (MATCHTYPE *)malloc(sizeof(MATCHTYPE))) == NULL) 
l'atal(“determine_nratch: creating MATCHTYPE, malloc\n”); 

match->Iine = m; 

match->angle_view_dilT = (m->est_angle_to_image_cemer - i->angle_to_image_center); 
inatch->ne\t = NULL; 

;* Place match m matchlist by increasing order of anglc_vicw_diff. */ 

if (i->matchlist == NULL) i->matchlist = match; 
else il'(labs(maich->anglc_view_diff) < fabs(i->matchlisL->anglc_view_diff)) 
i 

match->next = i->matchlist; 
i«>matchlist = match: 



place = i->matchlist; 

whilc((place->nexl != NULL) && (fabs(match->angle_view_diff) > 

fabs(place->next->angle_view_diff))) 

j 

i 

place = place->ncxt; 

) 

match->next = place->next; 
place->next - match: 



} /* end if( delta...) */ 
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/* */ 

/* void find_vcrtieal_rnatchc$(IMG_LINE *imgline, LINE *modeIIinc) 

/* 

/* Function to find all possible vertical model LINEs that could be matched 
/* to a vertical IMGJJNE. 

/* */ 
find_vertieal_matehes(imglinc, modelline) 

1MG_LINE *imgline; 

LINE *modellinc; 

( 



while (modelline != NULL) 

{ 

determine, vcrt_mateh(modeIline,imgline); 
modclline = modclline->NEXT; 



imglinc->pm = imglinc->matchlist; 



/* */ 

/* void print_matchlist(lMG_LIN 7 E *1) 

/* 

/* Function to ouipui the model LINEs matched to an IMGJJNE. 

/* V 
prini_matchlisi(J) 

IMGJJNE *1: 

l 

MATCHTYPE *m = I->matchlist; 

while(m != NULL) 

1 

printU” c: ( ,4f)”, m->line->namc, m->angle_vie\s_difQ; 
m = m->ne\t; 



printl'OViVU): 



/* V 

/* void declare_matches(IMG JJNE *1, IMG_L1NE *m, IMGJJNE *r, 

/* LINE *modellines) 

/* 

/* Declare all matches to model LINEs for left, middle, and right 
/* vertical lMGJJNEs. 

/* */ 

declare_maiches(l,m,r, modellines) 

IMGJJNE *1, *m, *r; 

LINE *modcllines; 

l 

find_vertical_matches(l, modellines); 
find_vertieal_matehes(m, modellines); 
find_vertieal_matches(r, modellines); 

printf(“\nlmage line matchlists to Model line NAME(angle difference): \n”) 

printf(“left (edge %sj >\n”,l->name); prim_matehlist(I); 
printf(“middle (edge %s) > VT,m->name); print_matehlist(m); 
printf(“rialu (cdne 9? s) > \n’\r->name); prini_matehlist(r); 

j 
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/* */ 

/* int determine_postionl(LlNE *1, LINE *m, LINE *r, double alpha 1, double alpha2, 
/* POSE *p, double *x, double *y) 

/* 

/* Routine to determine correct position based upon 3 matched model LINEs and 
/* the measured angle differences from the IMG_LINEs. Returns 1 if x,y 
/* position is determined, 0 otherwise. 

/* - */ 

int determine_position(l, m, r, alphal, aipha2, p, x, y) 

LINE *1, *m, *r; /* left, middle, and right viewed model LINEs */ 
double alphal, alpha2; /* measured image angles */ 

POSE *p; /* estimated (input) POSE */ 
double *\, *y; /* x,y location to be returned */ 

{ 

double chord ljength, chord2_lenglh, 

chordl_mid_x, chordl_mid_y, chord2_mid_x, chord2_mid_y, 

chord 1 .orient, chord2_orient, 

hi .orient, h2_orient, hi, h2, 

pvl.x, pvl.y, pv2_x, pv2_y, 

circle 1 .radius, circle2_radius, 

circle 1_\, circle l.y, circle2_x, circle2_y, 

c2c 1 , c2c 1 .orient, c2robot_orient, c2_omcga; 

/* Ensure that 3 different model LINEs are begin used. */ 
if ((I == m) II (in == rj II (I == r)) 



print! (“NO position: 2 LMG.LlNEs matched to same model LlNE\n”); 
return 0: 



/* Determine chord lengths, midpoints, and orientations. */ 
chord l.lenuth = 

sqrK ((1->M0DEL_X - m->MODEL_X )*(1->M0DEL_X - m->MODEL_X )) + 
((l->MODEL.Y - m->Mi *>EL_Y )*(1->M0DEL_Y - m->MODEL.Y ))); 



chord2_leneili = 

sqri(((m->MODEL_X - r->MODEL_X )*(m->MODEL.X - r->MODEL_X )) + 
Um->MODEL_Y - r->N10DEL_Y )*(n»MODEL_Y - r->MODEL_Y ))); 

if((chordl Jeneih < M1N.CHORD.LENGTH) 11 (chord2 .length < M 1 N.C H OR D.LEN GTH)) 

1 

prinLl ( ‘NO position: chord length < M1N.CHORD.LENGTHV’); 
return 0: 



chord l.mid.x = (I->MODEL_X + m->MODEL_X )/2.0; 
chord 1. mid.) = (1->M0DEL_Y + m->MODEL_Y )/2.0; 
chord2_mid.x = (m->MODEL_X + r->N10DEL_X )/2.0; 
chord 2_mid_> = (m->\10DEL_Y + r->MODEL_Y )/2.0; 

chord 1 .orient = map.am:le(atan2(l->MODEL_Y - m->MODEL_Y, 

1->M0DEL_X - m->MODEL_X)); 
chord2_orient = map_andc(aiun2(m->MODEL_Y - r->N10DEL.Y, 

m->MODEL_X - r->MODEL_X)); 

/* Determine orienLaLions perpendicular to the circle chords towards input pose. */ 
hl.orient = normalixe(chordl_orient + (Pl/2.0)); 
h2.orient = normalizc(chord2_orient + (Pl/2.0)); 
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/* Determine perpendicular lengths from the center of the chords to a point on the viewing circle. */ 
hi = (chord l_length/2.0) / tan(alpha 1/2.0); 
h2 = (chord2 Jength/2.0) / tan(alpha2/2.0); 

/* Determine the viewing points perpendicular to the chords. */ 

pvl_x = chordl_mid_x - (hi * sin(hl_orient)); 

pvl_y = chord l_mid_y + (hi * cos(hl_orient)); 

pv2_x = chord2_mid_x - (h2 * sin(h2_orient)); 

pv2_y = chord2_mid_y + (h2 * cos(h2_orient)); 

/* Determine radii and centers of viewing circles. */ 

circlel_radius = sqrt((hl*hl) + (chordl _length*chordl Jength/4.0))/2.0; 

circle2_radius = sqrt((h2*h2) + (chord2Jength*chord2Jengih/4.0))/2.0; 

circlel_x = pvl_x - (circle l_radius * sin(hl_orient + PI)); 
circle 1 _y = pvl_y + (circlel_radius * cos(hl_orient + PI)); 
circle2_x = pv2_x - (circle2_radius * sin(h2_orient + PI)); 
circle2_y = pv2_y + (circlc2_radius * cos(h2_orient + PI)); 

/* Determine distance and orientation between centers of viewing circles. */ 
c2cl = sqrt(((circlel_x - circlc2_x)*(circlel_x - circle2_x)) + 

((c ircle l_y - circle2_y)*(circlel_y - circlc2_y))); 



if(c2cl == 0.0) 



printfC'NO position: viewpoint circles identicaNr); 
return 0: 

) 

c2cl_orient = map_angle(atan2(circlel_y - circle2_y, circle l_x - circle2_x)); 
c2robot_orient = map_angle(atan2(p->y * circle2_y, p->x - circle2_x)); 

/* Calculate viewing position. */ 

if(circle2_radius < labs(c2c 1 - circle l_radi us)) c2_omega = 0.0; 
else if(circle2_radius > (c2cl -r circle l_radi us)) c2_omega = PI; 
else c2_omega = acos(((c2cl * c2cl) + (circle2_radius * circle2_radius) - 

(circle 1 _radius * circle l_radius)) / (2 * c2cl * circle2_radius)); 

if(left_of(c2roboi_oi ient, c2c 1 _orient)) 

( 

■x = circlc2_.\ - (circlc2_radius * sin(c2cl_orient+c2_omega)); 
x y = circle2_\ -t- (circlc2_radius * cos(c2cl_oricnt+c2_omega)); 

) 

else 

( 

*x = circle2_x - (circle2_radius * sin(c2cl_orient-c2_omega)); 

* v = circlc2_\ ■+■ (circle2_radius * cos(c2cl_orient-c2_omega)); 

) 

return 1; 

i 
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I* * j 

/* void sclect_next_hcst_match (MATCHTYPE *1, MATCHTYPE *m, 

/* MATCHTYPE *r, int *k) 

/* 

/* Select next best model LINE (minimum fabs(angle_view_diff)) for new 
/* combination in the next detcrmine_position() attempt. 

I* _ _ * j 

sclect_next_bcst_match(l,m,r,k) 

MATCHTYPE *1, *m, *r; /* MATCHTYPE pointers for left, middle and right 

IMGJLINES */ 

int *k; /* number of possible pm->next != NULL */ 



swiich(*k) 

( 

case 3: 

if((fabs(l->ncxt->angle_view_diff) < fabs(m->nexi->angle_view_diff)) 

&& (fabs(L>next->angle_view_diff) < fabs(r->next->angle_view_diff))) 

*k = 1; 

else if((l'abs(m->next->angle_view_difO < fabs(l->next->angle_view_diff)) 

&& (fabMm->next->anglejview_diff) < fabs(r->nexl->anglc_vicw_difT))) 

*k = 2: 
else *k = 3: 
break; 

case 2: 

ihj->nc.\t == NULL) 

1 

il(labs(m->ncxt->anglc_vic\v_dilf) < labs(r->next->angle_view_diff)) *k = 2; 
else *k = 3; 

) 

else i!(m->ne\t == NULL) 

i 

i 

il\lab.sU->next->angle_vicw_difO < fab$(r->ncxt->angle_view_difO) *k = 1; 
e Ne k — 3 1 



il(labs(L>next->angle_vicw_difO < fabs(m->next->angle_view_diff)) *k = 1 ; 

else *k = 2: 

t 

break: 

case 1: 

1 1(1 -> next != NULL) *k = 1; 
else if(m->ne\t != NULL) *k = 2; 
else *k = 3; 
break: 

default: 

*k = 0; 
break: 

) /* end switch */ 

} 
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/* */ 

/* double detcrmine_roiaLion (LINE *1, LINE *m, LINE *r, double x, double y, 

/* double iniheta, double loffang, double moffang, 

/* double roffang) 

/* 



/* Function that determines the average amount of rotational correction 
/* that is required based upon the 3 model LINEs angular offset from the 
/* IMG_LINEs to which they are matched with. 



/* 



7 



double dctermine_roiation(l, m, r, x, y, iniheta, loffang, moffang, roffang) 
LINE *1, *m, *r; 

double \, y, iniheta, loffang, moffang, roffang; 

{ 



/* calculate orientations of new position to Vertical Model LINEs */ 

double lneworicnt= map_angle(atan2(l->MODEL_Y - y, 1->M0DEL_X - x)), 
mneworient = map_angle(atan2(m->MODEL_Y - y, m->MODEL_X - x)), 
rncworieni= map_angle(atan2(r->MODEL_Y - y, r->MODEL_X - x)), 



/* calculate angles from center of image for new position */ 

lncwestangle = normalizeilneworicnt - iniheta), 
mnewestanglc = normalize(rnneworicnt - intheta), 
rnewcstangle = normali/.e(rncworicnt - intheta), 



/* calculate differences with offset angles from image */ 

l_diff = normalizc(lnewestangle - loffang), 
m_diff = normalize(mnewestangle - moffang), 
r_diff = normalize! rnewcstangle - roffang), 



/* average rotation difference */ 
rotation_dili = (l_diff + m_diff + r_diff) / 3.0; 



return rotation_dill: 

} 
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/* V 

/* int verify_pose(double x, double y, double theta, IMG_LINE *ImgLines, 

/* LINE * ModelLines) 

/* 

/* Funciion lo relurn number of IMG_LINEs lhai lie over model LINEs 
/* for a corrected 2D view of the model environment for a given possible pose. 

I* * j 

int verify_pose(x,y,theia,ImgLines,ModelLines) 
double x,y, theta; 

IMG_LINE MmgLines; 

LINE *ModelLines; 

{ 

LINE *ml = ModelLines; 

IMG_LINE *il = lmgLines; 
double delta; 

int endpoint_inclusion = 0, imgline_on_modeIline = 0, nr_good_hits = 0, 

found = 0; 

/* Calculate est_angle_toJmage_center for all ModelLines based upon 
pose: (x,y,theia). */ 
while (ml != NULL) 

{ 

ml->est_angle_to_image_cemer = 

normal ize(map_angle(aian2(ml->MODEL_Y-y, ml->MODEL_X-x)) - theta); 
ml = ml->NEXT: 

} 



/* Find correspondences of image lines to model lines. V 
whilc(il != NULL) 



r Ensuie vertical image line is not due to image borders. */ 
if( labs(il->anele_to_imauc_cenier) < FIELD_H ALF_ANGLE) 

i 

/* Find a corresponding IMG_LINE. */ 
ml = ModelLines; 
found = 0; 

while ((ml • = NULL) && Hound) 

( 

delta = (ml~>Yl - il->pl.y) + (il->p2.y - ml->Y2); 

endpointjnclusion = 

((delta <= 0.0) II 

((delta > 0.0) && (delta/il->dmajor < DELTA_LENGTH_RATIO))); 

imgline_on_modelline = 

(fabs(iI->angle_ioJmage_center - 

ml->cst_anglej,oJmage_center)<VERIFY_EQ_ANGLE_EPSILON); 

il'(cndpoint_inclusion && imgline_on_modelline) 

{ 

++nr_good_hits; 
found = 1 ; 

1 



ml = ml->NEXT; 

) 

) 

ll = il->next: 



return nr_good Juts; 

i 
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/* */ 

/* POSE *calculate_bcsi_pose(IMG_LINE *1, IMG.LINE *m, 1MG.LINE *r, 

/* double alpha_lm, double alpha_mr, POSE *in_pose, 

/* IMGJJNE *lmgLines, LINE *ModeILines) 

/* 

/* Function that conducts several iterations of finding an 1MG_LINE to LINE 
/* triplet, determines the POSE based upon the matching, and returns the best POSE 
/* based upon number of IMG JL1NE to model LINE matches of a 2D view of the 
/* best POSE and iranslaLional error of the robot. 

/* V 



POSE *culculaie_best_pose(l,m,r,alphaJm,alpha_mr,rn_pose,lmgLines,ModelLines) 
IMGJJNE *1, *m, *r, *ImgLines; 
double alpha_lm, alpha_mr; 

POSE *in_pose; 

LINE *ModclLincs; 



l 

int 

POSE 

double 

MATCHTYPE 



I = 0, k = 0, nr_hits = 0, best_nr_hiis = 0; 
* bcst_pose; 

\, y, rot, trans = 0.0, best_rot, besijrans: 
Mmatch = l->maichlist, 

*mmateh = m->matchlist, 

Ninaich = r->maichlist; 



printlV'Pose detemunaiion: \nj; 

if((best_pose = (POSE ■ )malloc(sizeof(POSE))) == NULL) 
ratal(“calculaie_bcst_posc: mallocVC); 



if((lmaich == NULL) li (mmaich == NULL) Il(rmatch == NULL)) 

l 

primir’Can not determine pose for %s, %s, 9ts (NULL maichlist)\n’\ 
l->name,m->namc,r->rramc): 

J 



else lory = 0; j < 20; ++j j 



prinih" |U.s,U s.9r sj “,lmatch->line->name,mnuuch->line->namc, 
rmaich->line->namc); 

ii (determine j)osition(lmatch->line, mmatch->line, rmatch->line, 
alphajm, alphajnr, in_pose, &x, &y)) 



/* Calculate translation and rotation differences. */ 

tran» = sqri((in_posc->x - x) * (in_pose->x - x)) + 
((in_pose->y - y) * (in_pose->y - y)); 



rot = deiermine_rotalion(lmaich->line, mmaich->line, rmatch->line, 
x, y, in_pose->theta, 
l->angle_toJmage_center, 
m->angle_to_image_centcr, 
r->anglejoJmage_center); 

nrjuh = vcrify_pose(x,y,in_pose->theta + rot,lmgLines,ModelLines); 



printf(“x= %.2f y= %.2f T= %.4f R= %.4f nr verify matchcs= %d\n”, 
x,y,tran$,rot,nr_hits); 

if(G == 0) II (((doublc)nr_hits)/trans > ((double)best_nr_hits)/best_trans)) 

( 

best_nr_hits = nr_hits; /* save best irans and rot values */ 
best_trans = trans; 
best_rot = rot; 

best_pose->x = x; /* assign values to bcst_pose */ 
best_pose->y = y; 

best_pose->theta = in_pose-> theta + rot; 
printfC* *** BEST POSE MODIFIED ***\n”); 
l->pin = lmatch; /* set 3 IMG__LINE’s match pointers */ 
in->pm = mmatch; 
r->pm = rmatch; 



J 

/* Select the next best match combination of model LINEs. */ 

k = ((lmatch->next != NULL) + (mmatch->ncxt != NULL) + 
(rmatch->next != NULL)): 

il(k > ()> 



^elcct_ncxt_best_match(lmatch, mmatch, rmatch, ck:k); 
if(k == 1) lmatch = lmatch->next; 
else if(k == 2) mmatch = mmatch->next; 
else if(k == 3) rmatch = rmatch->ncxt: 

) 

else return lvst_pose: 



return best_pose: 

) 
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/* */ 

/* POSE *update_posc(IMG_LINE *ImagcLines, LINE * Model Lines, POSE *in_pose) 
/* 

/* Function to return a corrected POSE given the vertical edges extracted from an image 
/* and the vertical model LINEs that should be visible from the expected POSE of the 
/* robot. 

/* */ 

POSE *updatc_pose(ImageLines,ModeLLines,in_pose) 

1MG_LINE *lmageLincs; 

LINE *ModclLines; 

POSE *in_pose; 

{ 

IMG_LINE *il, ^ lei t, ^middle, *right; 

POSE *out_pose; 

double alphajm, alpha_mr; 

int j: 



if((out_pose = (POSE *)malloc(sizeof(POSE))) == NULL) 

laLil( fc ‘update_pose: creating POSE out_pose: mallocNn”); 



/* Find the 3 most significant IMG JJNEs and return them in 
left-to-right order pointed to by *left, ^middle, and *riglu. */ 



d = ImageLines; 

printl'CViDetcnnine 3 major image edges for matching: — -\n”); 



for(j = 1: j < 4;) 

{ 

ih.il == NL LL • fata I (.“CAN NOT DETERMINE POSE: < 3 image lincsNn”); 
* Ensure wrtical image line is not due to image borders. */ 
iHlabAil->angle_io_image_center) < F1ELD_H ALF_ANGLE) 



iUj== U 

i 

left = il; 

printf(“ 9ld) line %s, angle from center = %.4N\'\ j, 
iL>name, iL>anglc_to_imagc_center); 

++j: 



else il\(j == 2) && (iabs(il->angle_to_image_center - 

left->angle_to_image_center) > IMG_LINE_MIN_ANGLE)) 

( 

i 1 (jI->ande_to _image_center > lei t->angle_to_image_cenier) 

1 

right = left; left = il; 

} 

else right = il; 

printf(“ %d) line %s, angle from center = %.4l\n”, j, 
il->name, il->angle_loJmagc_cemcr); 

] 
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else if(G ==3) && 

(fabs(il->angle_to_image_center - 

left->angle_toJmage_center) > IMG_LINE_MIN_ANGLE) && 
(fabs(il->angle_toJmage_centcr - 

right->angle_to_image__center) > IMG_LINE_MIN_ANGLE)) 

( 

il(il->angle_to_image_ccnter > lclt->angle_to_image_center) 

l 

middle = leU; left = il; 

} 



else if(il->angle_to_image_centcr < righi->anglc_toJmage_center) 

( 

middle = right; right = il; 

) 



else middle = il; 

prinil'(“ %d) line %s, angle from center = %.4f\n”, j, 
il->namc, il->angle_toJmagc_cenier); 



-h-j: 



il = il->ne.\i; 



/* Calculate two angles between the three major IMGJJNEs. */ 

alpha_lm = left->angle_to_image_centcr - middle->anglc_to_image_center; 
alpha_mr = middle->angle_to_image_center - right->anglc__to_image_ccntcr; 

/* Declare the matches to model LINEs for the three major 1MG_L1 NEs. */ 
dcclare_maichesileft,middle,nght,ModclLines); 



/* Calculate the correct pose. */ 

out_pose = calculateJxst_pose(left, middle, right, alpha_lm, alpha_mr, 

in_pose, ImageLines, ModclLines); 



return out_pose: 
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C. FILE: MATCHDISPLA YSUPPORT.H 



^* ******************************************************************** 
/* FILENAME: matchdisplaysupport.h 
/* AUTHOR: Kevin Peterson 
/* DATE: 07 February 1992 



/* 

/^DESCRIPTION: Collection of display functions. 
P 



/* void display_matchjmagc (NPSIMAGE *img, IMG_LINE *1, 

/* w ' LINE_HEAD *ml, LINE_HEAD *m2) 

/* void draw_whitc_lincs(IMG_LINE *1) 

/* void draw_white_modeLlines_with_names (LINE * m) 

/* void draw_whitc_model_Jines (LINE *m) 

/* void draw_black_modcl_lines_with_names (LINE *m) 

/* void draw_black_model_lines (LINE *m) 

/* void draw_match_lines (1MG_LINE *1) 

/* void displayjnatchjoop (NPSIMAGE *img, IMG_L1NE *1, 

/* LINE_HEAD *ml, *LINE_HEAD *m2, long winid) 

p 



P These functions make calls to following SiliconGraphics routines: 

/* prcfsizeQ, winopenQ, winsetQ, winclose(), RGBmodeQ, 

/* singlcbuflcrO, gconfig(), qdevice(), c3f(), move2(), draw2(), 

/* swapbuffers(), IrectwriteQ, reshapevicwport(), cmov2(), 

/* charsir(), and clcar(). 

/* ******;*. ^ * ^S*;****^;*;******;^****;*:;*;*;*:***:*::***:*::*})*********** 



/•— V 

/* void display_match_imagc (NPSIMAGE *img, IMG_L1NE *1, 

/* LINE_HEAD "ml, LINE.HEAD *m2) 

/* */ 

display_maich_imagc(imgj,ml ,m2) 

NPSIMAGE *img; /* input image */ 

1MG_L1NE "1; /* list of IMG_LINEs found from image */ 

LINE_HEAD > ml , m2; /* LINE_HEAD types from Jim’s “graphics. c” */ 

i 



long winid; 



prcfsizc(img->xsize,img->ysizc); /* preferred size for window */ 
winid = winopen(img->name); /* open the window */ 
RGBmodeO; /* set RGBmode, singlebuflcr, and */ 
singlcbuffcr(); p configure the window */ 
gconfigQ; 



/* initialize_controls */ 

qdcvicc(REDRAW); 

qdevice(LEFTMOUSE); 

qdcvicc(MIDDLEMOUSE); 

qdcvice(RIGHTMOUSE): 

qdevicc(ESCKEY): 



display_matchJoop(img,l,ml, m2, winid); 

free(img->imgdata.bitspu); /* delete the bitmap for the image */ 
free(img); /* delete the NPSIMAGE structure */ 
winclosc(winid); /* close the window */ 

) 
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/* 

/ * void draw_w hiie_lines(lMG_LlN T E *1) 

/* 

/* Draws while lines over an image. 

/* */ 

draw_white_lines(l) 

IMG_L1NE *1; 

( 

sialic floal whiic[3J = { 1.0, 1.0, 1.0); /* rgb while */ 

c3f(whiie); 

while(l!=NULL) 

( 

cmov2(l->pl.xJ->pl.y); charsir(l->name); 
move2(l->pl.x,l->pl .y); draw2(l->p2.x,l->p2.y); 
1 = l->nc\l: 

} 

swapbulier.s(j; 

} 



I* */ 

r void dra\v_whiiejnodel_lines_wiih_names (LINE *m) 

I* 

r Draws blue model lines over an image. 

r */ 

draw_\\ hiie_model_line\_vs uh_names(m) 

LINE *m: 

l 

sialic lloai while] 3 1 = 1 1 .0,1. 0,1.0); /* rgb while */ 

c3f(whiie): 

while(m!=NLLLj 

( 

cinov2(n»Xl,m->Yl j; charsir(m->name); 
move2(m->X 1 .m-> Y 1 ); draw2(m->X2,n»Y2): 
in = m->NEXT: 

} 

swapbullersi ): 



/* */ 

/* void dra\\_black_model_lines_w'iih_names (LINE *m) 

/* 

/* Draws black model lines over an image. 

r - */ 

draw_black_model_lines_wiih_names(m) 

LINE *m: 

{ 

sialic lloai blaek| 3 1 = 10.0,0.0,0.0); /* rgb black *1 

c3f(black); 

while(m!=NLLL) 

{ 

cmov2(m->X 1 ,m-> Y 1 ); char$ir(m->name); 
move2(rn->Xl,m->Yl); draw2(m->X2,m->Y2); 
m = m->NEXT; 

) 

swapbulfcrsQ: 

} 
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/* 

/* void draw_white_model_lines (LINE *m) 

/* 

/* Draws while model lines over an image. 

/* */ 

draw_whiie_modclJines(m) 

LINE *m: 

{ 

static float whiie[3] = { 1.0.1. 0,1.0}; /* rgb white */ 

c3f(whiie); 

whilc(m!=NULL) 

{ 

inovc2(n»Xl,m->^ 1); draw2(m->X2,m->Y2); 
m = m->NEXT: 

} 

swapbulIersO: 

} 



r */ 

/* void dra\\_blaek_niodel_lines (LINE *m) 

/* 

/* Draws black model lines over an iimuzc. 

/* - *7 

draw_blaek_model_lineMm ) 

LINE *m: 

( 

sialic float blaekl 31 = ; 0.0, 0.0, 0.0} ; /* rgb black */ 

c3l (black); 
while(m!=NULL) 

i 

mo\e2(m->Xl ,m->Y 1 ): draw2(m->X2,m->Y2): 
m = m->NEXT; 

i 

swapbullersQ: 

) 
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/* */ 

/* void display_match_loop (NPSIMAGE *img, IMGJJNE *1, 

/* LINE_HEAD *ml, LINE_HEAD *m2, long winid) 

/* */ 
display_maich_loop(img,l,ml,m2,winid) 

NPSIMAGE *img; /* input image */ 

IMG_L1NE *1; /* list of lMG_LINEs found from image */ 

L1NE_HEAD *ml, *m2; /* L1NE_HEAD type from Jim’s “graphics.c” */ 
long winid; 

{ 

sialic float white[3] = { 1.0, 1.0, 1.0); /* rgb white */ 
short value; /* value returned from the event queue */ 



/* display the images once */ 



winset(winid); 

lrectwritc(0,0,img->xsi/e - l,img->ysize - l,img->imgdata.bitsptr); 
draw_whiicjincs(l); 



/* loop until a mouse button is pressed */ 
whilc(TRUE) 

{ 

s witch(qrcad(& value ) ) 

{ 

case REDRAW': 

winsct((long)valuc); 
reshapevicwportO; 
if(value == winid) 

{ 

lrectw rite(0,0,img->xsize - I,img->ysize - 1, img->imgdaia.bitsptr); 
dra\\_whitcjines(l); 

1 

break; 

case LEFTMOUSE: 
if( value == 0) 

1 

lrcct\M'ite(0,0,img->xsize - l,img->ysize - 1, img->imgdata.bitsptr); 

dra\\_white_modclJincs_with_namcs(ml->VLlNE_LIST); 

dra\\_white_modclJincs(ml->LINE_LIST); 

) 

break; 

case MIDDLEMOUSE: 
if( value == 0) 

( 

c3f(w hite); 
clearQ; 

dra\^_black_model_lincs_with_names(ml->VLlNE_LlST); 

draw_black_model_lines(ml->LINE_LlST); 

1 

break: 

case R1GHTMOUSE: 
if( value == 0) 

( 

lreciwritc(0,0,img->xsize - l,img->ysizc - 1, img->imgdata.bitspir); 
dra\\_whitc_model_lincs(m2->VLINE_LIST); 
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dra\\_white_modelJines(m2->LINE_LIST); 

) 

break; 

case ESCKEY: 
return; 

default: 

bre;tk: 

} /* end switch */ 

/* end while */ 



129 



APPENDIX D- USER’S GUIDE 



file: README 
date: 8 March 92 
by : Kevin Peterson 

USER'S MANUAL FOR YAMABICO' S VISIUAL NAVIGATION ROUTINES 

This file outlines the functions of the following programs: 
findedge, fastedge, and vertmatch 

1. REQUIREMENTS: 

1.1 HARDWARE: The above routines were developed on a 
SiliconGraphics Personal Iris workstation and can be run on 
all SGI stations at NPS. 

1.2 SOFTWARE: Ensure the fcllowing files are in the 
local directory for the edge extraction routines: 

README (this file) 

Makefile 
image_types . h 
match_types . h 
np si mage support . n 
edgesupport . h 
vert support . n 
displaysupport . r. 
f indedge . c 
fastedge . c 

Ensure tne following files are in the local directory for the 
matching pose determination routines: 

5th. h 
2d-r . h 

graphics . n 

mat chdi splay support . h 
matchsuppo rt . r. 
vertmatch . c 



2 . COMPILATION: 

2.1 The f D_1 owing C program libraries are required: 
device . n 

math . h 
stdio . h 
gl . h 

gl/ image . L 

2.2 findedge, sasedge, fastedge, fastsobel, vertedge, 

and vertmatch car. all be compiled by the command line similar to 
cc -o f indedge f indedge. c -limage -lgl_s -lm 
cc -o fastedge fastedge. c -limage -lgl_s -lm 
etc . . . 

or if the Makefile is present: 
make f indedge 
make fastedge 
etc . . . 
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3. PROGRAM FUNCTIONS: 



3.4 FINDEDGE - Displays the grayscale and gradient 
images of the input image. Conducts edge extraction based 
upon least squares fit. 

3.6 FASTEDGE - Displays the input image and edges 
extracted. Uses the green component of the RGB pixel value 
vice the grayscale value and does not create the grayscale or 
gradient images. 

3.9 VERTMATCH - Extracts vertical edges from the input 
image (VERTEDGE) , creates a 2D view of the model environment 
based upon the estimated pose (dead-reckoning maintained 
configuration) of the robot, conducts line matching between 
image vertical edges and model vertical line segments, 
determines possible poses based upon different matchings for 
the three major image edges, then returns the best pose based 
upon number of edge to model line matches and distance of 
updated pcse t: the estimated (input) pose. 



4. USING THE PROGRAMS : 

4.4 findedge < filename > 

Filename is input picture file to extract edges from. Two 
windows will be created; one for the grayscale image, the 
other for the gradient image. 

- Lines extracted can be drawn onto either image by the 
LEFT or MIDDLE mouse buttons while the cursor is 

in tne TI?LE_BAE . 

- Lines can re irawn ever a white background by MIDDLE 
mouse button whole the cursor is in the WINDOW. 

- Exit: RIGHT or LEFT mouse buttons (cursor in WINDOW) or escape key. 

4.6 fastedge <filename> 

One window is opened to display the input (color) image with 
extracted edges drawn over tne image. 

- Lines extracted can be drawn onto either image by the 
LEFT mouse buttons while the cursor is in the WINDOW. 

- Lines can oe dra v;n ever a white background by MIDDLE 
mouse button while tne cursor is in the WINDOW. 

- Exit: LEFT mc^se buttons (cursor in WINDOW) or escape key. 



4.9 vertrnatch <filename> 

One window is opened to display the input image with 
the extracted vertical edges. 

- MIDDLE mouse button (while in WINDOW) displays the 2D wire-frame 
view for the expected pose over a white background. 

- RIGHT mouse butter, displays 2D expected pose wire-frame view over 

the 

input picture. 

- LEFT mouse button displays 2D wire-frame view from the corrected 

pose 

over the input picture. 

- Escape key exits. 
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